JP3066594U - Image conversion device - Google Patents

Abstract

(57) [Problem] To provide an image conversion device capable of reducing the number of monitoring cameras installed and contributing to cost reduction. An image conversion device (10) for converting a fish-eye image captured using a fish-eye lens (2) into a plane image for display, comprising: a cut-out unit (10a) for cutting out a predetermined number of image regions from a fish-eye image; An image conversion unit 10b that converts a fisheye image in an image region cut out by the unit 10a into a plane image, and an image output unit 10c that outputs the converted plane image to the frame switcher device 11.

Description

[Detailed description of the invention]

[0001]

[Technical field to which the invention belongs]

 The present invention relates to an image conversion device that converts a fisheye image captured using a fisheye lens into a planar image for display.

[0002]

[Prior art]

 Surveillance cameras are installed in stores and the like, and images taken by the surveillance cameras are recorded or displayed on a monitor for monitoring. In this case, the area captured by one surveillance camera is not enough, so multiple surveillance cameras are installed, these are connected to a frame switcher device, and the monitor screen is divided into a Images from the camera were projected simultaneously.

[0003]

[Problems to be solved by the invention]

 However, installing a plurality of surveillance cameras is costly. In addition, changing the area captured by the surveillance camera requires changing the camera mounting position and a mechanism to move the camera, resulting in cumbersome work and further cost increase. I was

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image conversion apparatus capable of reducing the number of monitoring cameras installed and contributing to cost reduction.

[0005]

[Means for Solving the Problems]

 In order to achieve the above object, an image conversion device according to the present invention is an image conversion device that converts a fisheye image captured using a fisheye lens into a plane image for display, and a predetermined number of images from the fisheye image. A cutout unit that cuts out an image region, an image conversion unit that converts a fisheye image in the image region cutout by the cutout unit into a flat image, and an image output unit that outputs the converted flat image to a frame switcher device. It is characterized by having.

The operation of this configuration is as follows. Instead of using a standard lens or a normal wide-angle lens as a lens to be attached to the surveillance camera, a fisheye image is shot using a fisheye lens. Although a fisheye image can capture a wide area, it is difficult for the human eye to recognize it even if it is displayed on a monitor as it is because of the distortion peculiar to the fisheye image. Therefore, the fisheye image is converted into a plane image. A predetermined number of image regions are cut out from the fisheye image, and the cutout image region is converted into a plane image.

That is, a predetermined number of planar images can be extracted by cutting out a predetermined number of image regions from one fisheye image. Therefore, a predetermined number of planar images having different fields of view can be extracted from one surveillance camera, so that substantially one surveillance camera performs the same function as a plurality (predetermined number) of surveillance cameras. By outputting a predetermined number of planar images having different fields of view to the frame switcher device, it can be displayed on a monitor. As a result, it was possible to provide an image conversion device that can reduce the number of monitoring cameras installed and contribute to cost reduction.

[0008] As a preferred embodiment of the present invention, an image region cut out by the cut-out section can be changed. According to this configuration, the field of view can be changed by changing the image area to be cut out. Compared to the conventional technology in which the position of the surveillance camera is changed or the surveillance camera is moved by the drive mechanism, the task of changing the field of view can be simplified and contribute to further cost reduction. .

In another preferred embodiment of the present invention, the clipping section circulates and clips the predetermined number of image areas at a predetermined cycle time. As a result, images from a plurality of surveillance cameras can be substantially acquired at a predetermined cycle time.

In still another preferred embodiment of the present invention, the image conversion unit calculates a first coordinate for projecting coordinates of the plane image onto a virtual object plane of a fisheye image plane. And a second coordinate operation unit for obtaining a second projection coordinate obtained by projecting the first projection coordinates obtained by the first coordinate operation unit onto the fisheye image plane.

According to this configuration, first, the first coordinate calculating unit obtains first projection coordinates obtained by projecting the coordinates of the plane image onto the virtual object plane of the fisheye imaging surface, and then calculates the first projection coordinates by the fish. The second projection coordinates projected on the eye imaging surface are obtained. In other words, coordinate conversion is performed step by step in accordance with the procedure of coordinate conversion instead of formulating all at once when performing image conversion. This makes it possible to configure an arithmetic unit by combining simple arithmetic circuits, and to reduce the cost of the hardware of the image conversion device.

[Embodiment of the invention]

 <System Configuration Example> FIG. 1 shows a configuration of a monitoring system including the image conversion device according to the present invention. This system includes a surveillance camera 1 equipped with a fish-eye lens 2 and a fish-eye converter 10 (corresponding to an image conversion device) that obtains a predetermined number of planar images from the fish-eye lens captured by the surveillance camera 1 by performing image conversion. , A frame switcher device 11, a monitor 12, a VTR 13, and a changeover switch 14.

The place where the surveillance camera 1 is installed is not particularly limited, and is installed, for example, in a store or a parking lot. For example, if it is installed so as to hang from the ceiling in the store, the entire area in the store can be captured by the fisheye lens 2.

The fisheye converter 10 includes a cutout unit 10a, an image conversion unit 10b, and an image output unit 10c. First, the cutout portion 10a will be described. FIG. 2 shows the entire fish-eye image I captured by the fish-eye lens, and the central circular portion is a substantial image. From this fisheye image I, four (corresponding to a predetermined number) image areas G1, G2, G3, and G4 are cut out. Which part is cut out can be set and input. For example, the north side of the store is cut out as the image area G1, the east side as the image area G2, the south side as the image area G3, and the west side as the image area G4.

Since the fish-eye image I is difficult for a human to understand when displayed on the monitor 12 as it is, a plane image of each of the cut-out regions G1 to G4 is obtained. A method of converting a fisheye image into a plane image will be described later. The converted two-dimensional images are transferred to the frame switcher device 11.

In the cut-out section 10a, the cut-out area is circulated and switched in the order of G1, G2, G3, and G4. When the monitoring camera 1 is of the NTSC system, the image data sent from the monitoring camera 1 is stored in the frame memory (see FIG. 8 described later), and the image data in the frame memory is updated every 1/30 second. You. Therefore, when four cutout areas are set, the time from cutting out one image area to cutting out the next image area is 1/30 seconds ÷ 4 = 1/120 seconds.

The image output unit 10 c has output terminals corresponding to the respective cutout areas, and outputs image signals G 1 to G 4 according to the NTSC system to the frame switcher device 11. The frame switcher device 11 has a known structure. For example, a frame switcher manufactured by Matsushita Electric or a multiplexer manufactured by Sony can be used. As can be seen from FIG. 1, the frame switcher device 11 receives image signals G1 to G4 from the input terminals and is in the same state as if four surveillance cameras 1 were connected. Only one surveillance camera 1 is used.

Here, the image recorded on the VTR 13 is a fisheye image (live image) before image conversion, which is shown in FIG. FIG. 3 is a diagram conceptually shown, but is recorded on the video tape T in the state of a fisheye image every 1/60 second. On the other hand, when images are frame-multiplex-recorded by the frame switcher device 11 using four surveillance cameras, the images G1 → G2 → G3 → G4 every 1/60 second as shown in FIG. Are recorded cyclically in this order. In other words, in the case of frame multiplex recording, the number of frames to be recorded is reduced to 1/4 in the example of FIG. 3 (thinned out). There is an advantage that the number of frames can be quadrupled when compared with recording. Also, if the number of frames per second is set to be the same between the case of recording with a fisheye image and the case of recording with frame multiplex recording, recording with a fisheye image is 1/4. Since time is required, the number of video tapes can be reduced to 1/4, which is advantageous in terms of cost.

As shown in FIG. 4, each of the image areas G 1 to G 4 cut out from the fish-eye image I is converted into a plane image and sent to the frame switcher device 11. Reference numeral 12 divides the display screen into four and displays each image area G1 to G4 as a plane image.

When the fisheye image I of the VTR 13 is recorded and the changeover switch 14 (see FIG. 1) is switched to the reproduction side for reproduction, the image areas G1 ′, G2 ′, G3 ′, G4 ′ to be cut out are obtained. Can be changed and displayed on the screen of the monitor 12. As shown in FIG. 2, when the image areas G1 to G4 are cut out, not all the range that can be monitored by the monitoring camera 1 is reproduced as a planar image. Therefore, if a different location can be cut out and reproduced as a two-dimensional image, monitoring without leakage can be performed. That is, the image areas G1 to G4 can be displayed in real time, and the other image areas G1 'to G4' in the same time zone can be confirmed later.

In the above embodiment, the number of image areas to be cut out is four, but is not limited to this.

<Configuration Example of Image Conversion Unit> Next, an example of a method of converting a fisheye image into a planar image will be described. Figure 5 is a diagram showing a setting of a coordinate system, FIG. 6 is a diagram for explaining the correction coefficient k _1, 7 is a diagram for explaining a procedure for converting coordinates by hardware, 8 is mounted on fisheye converter It is a figure showing a circuit block.

<Positional Relationship of Coordinate System> Referring to FIG. 5, the positional relationship of coordinates to be transformed will be described first. The coordinate system is set as follows. First, as a space for indicating the position of a subject, an (x, y, z) coordinate space in which the position of the fisheye lens is the origin and the direction of the optical axis is the z axis is considered. Also consider the azimuth (角) and the zenith angle (θ) as parameters indicating the position of the subject as viewed from the origin.

The position of an object photographed by a fish-eye lens on an image sensor (such as a CCD) is determined by the angle viewed from the fish-eye lens. Therefore, it is assumed that the object is located on a hemispherical surface having a radius of one. Note that this hemisphere is called a virtual object plane. The plane image desired to be obtained as a result is shown in the (u, v) coordinate system in FIG. 5, and the center (origin) of the (u, v) coordinate system is represented by the (x, y, z) coordinate system. It is assumed that it is located at a distance of 1 from the origin and is in contact with the above-mentioned hemisphere which is the virtual object plane. The plane represented by this (u, v) coordinate system is associated with the pixels of the display image on the monitor. Note that an angle between the u-axis of this coordinate system and the (x, y) plane, more specifically, a plane passing through the origin of the (u, v) coordinate system and parallel to the (x, y) plane, and (u , V) The angle between the line of intersection with the plane and the u-axis is (α).

An image plane (fish-eye image) obtained through a fish-eye lens is represented by a (p, q) coordinate system as shown in FIG. The (p, q) coordinate system has a positional relationship parallel to the (x, y) plane, and its origin is on the z-axis. At the position on the imaging surface (for example, the position of the pixel on the CCD imaging device), the image circle diameter differs depending on the size of the imaging device and the focal length of the fisheye lens. = 0), the fisheye image is projected on a circle with the radius of the image projected on the subject being 1, and the magnification is adjusted in actual use.

In the image conversion, a point on the plane image (u, v coordinates) is calculated by calculating on which position on the imaging surface (p, q coordinates) the image is to be obtained, and luminance information of the point is referred to. Thus, data of a planar image to be obtained as a result can be generated. Information on the viewpoint (ψ, θ, α) and the enlargement ratio (zoom magnification), which is the size of the plane image, is information that is input by the operator via a keyboard, a pointing device, or the like. It is obtained and set as data for calculation by the processing unit of (1).

First, as shown in FIG. 5, necessary parameters indicate the center (origin) of the planar image (x ₀ , y ₀ , z ₀ ); When one pixel is moved in the direction (corresponding to one pixel on the monitor screen), the change in each axis direction at the (x, y, z) coordinates is represented by {ux, {vx,} uy, ∂vy, ∂uz, ∂vz; These parameters can be easily obtained from the above-mentioned viewpoint angle information (ψ, θ, α) and information on the magnification of the image.

<Calculation Procedure for Image Conversion> Next, a calculation procedure for image conversion will be described with reference to FIG. The calculation procedure is roughly divided into two steps, Step 1 and Step 2.

(Step 1) First, the coordinates of a point P ′ (first projected coordinate) in which a point P on the planar image (on the u and v coordinates) is projected on a hemisphere which is a virtual object plane are obtained. . X of the point P, y, z coordinates _{(x 1, y 1, z} 1), u, v coordinates When _{(u 1, v 1),} x 1 = x 0 + u 1 · ∂ux + v 1 · ∂vx y ₁ = y ₀ + u ₁ · ∂uy + v ₁ · ∂vy z ₁ = z ₀ + u ₁ · ∂uz + v ₁ · ∂vz. In addition, as described in FIG. 6, the point P ′ is on the line connecting the OPs, so that a certain coefficient is k ₁ and (x ₂ , y ₂ , z ₂ ) = k ₁ ( _{x 1, y 1, z 1} ) relationship is established (see Figure 6). Then, it is expressed by the origin of the (x, y, z) coordinate system, the origin (x ₀ , y ₀ , z ₀ ) of the (u, v) coordinate system, and the coordinates (x ₂ , y ₂ , z ₂ ). The distance of the point P ′ is 1. Therefore, the distance L of the point P from the origin of the (u, v) coordinate system is L = (u ² + v ² ) ^0.5 . For k _1, since a constant if Kimare the distance L is, the table of k ₁ to pair with the distance L previously created as a look-up table, by multiplying the k ₁ determined from the look-up tables, As described above, (x ₂ , y ₂ , z ₂ ) can be obtained. In other words, as shown in FIG. 7 is a _{x 2 = k 1 · x 1} y 2 = k 1 · y 1 z 2 = k 1 · z 1. Thus, the first projection coordinates on the hemisphere were obtained.

(Step 2) Next, as a second stage of the calculation, the first projection coordinates (x ₂ , y ₂ , z ₂ ) obtained previously and the second projection coordinates W (p ₁ ) on the fisheye imaging plane are calculated. , Q ₁ ). See FIG. 5 for W. As has been already described, 'since that is on the hemispherical surface of radius 1, the point P' the point P zenith angle (theta ₁₎ is uniquely determined from the value of z ₂ for (theta _1, as shown in FIG. 5). Therefore, θ ₁ = cos ⁻¹ (z ₂ ) (1). Further, since the azimuth angle of the point W on the imaging surface and the point P ′ on the hemisphere are the same, (p ₁ , q ₁ ) = k ₂ · (x ₂ , y ₂ ) (2) ) Holds. Since the projection type fisheye image, from the origin of the fish-eye imaging plane (p, origin of q coordinate system), to the point W height (h) which is I table as a function of theta _1, h = F (θ ₁ ) (3) Here, as an example of some specific functions of F (θ ₁ ) according to the projection method, assuming that the focal length of the fisheye lens is f, equidistant projection: h = f · θ stereoscopic projection: h = 2f · tan (θ / 2)

By substituting equation (1) into equation (3), h = F (cos ⁻¹ (z ₂ )) (4), which can be expressed as a function determined by z ₂ . The distance r from the origin of the (x, y, z) coordinate system of the point Q, which is obtained by projecting the point P ′ on the (x, y) plane, is given by the fact that the point P ′ is a point on a spherical surface having a radius of 1. , R = (1−z ₂ ² ) ^0.5 (5) Therefore, the equation (4) (5) _{than, k 2 = h / r =} F (cos -1 (z 2)) / (1-z 2 2) 0.5 ··· (6) that is, the formula (2) coefficient k ₂ of the can be determined as a function of z _2. Therefore, for the coefficient k ₂ , a look-up table is created from the value of z ₂ according to equation (3), and the second projection coordinates (p ₁ , q ₁ ) on the imaging plane are found using the value. be able to. That is, p ₁ = k ₂ · x ₂ q ₁ = k ₂ · y ₂ .

As described above, in the first step for image conversion, the operations are limited to addition, subtraction, multiplication, and square root, and the other functions correspond to the lookup table. Further, in the second step, the operation is limited to multiplication, and other function calculations and the like also correspond to the lookup table. Then, by performing each operation step by step, a simple arithmetic circuit (logic circuit such as an adder) can be configured in a pipelined manner. In addition, the portion for performing complicated function calculations can be reduced in circuit scale by supporting a lookup table. Further, by changing the table to determine the k ₂ in step 2, can be easily, moreover correspond by the same arithmetic circuit fisheye different projection schemes.

<Example of Circuit Configuration> Next, a specific circuit block configuration in the fisheye converter 10 will be described with reference to FIG. A fish-eye lens 2 is mounted on the CCD camera 1, and image information taken by the CCD camera is sent to a fish-eye converter 10 and processed for image conversion. The fisheye converter 10 includes a camera interface 30 that captures fisheye image data from the CCD provided in the CCD camera 1, a frame memory 31 that stores two frames of fisheye image data, An interpolation calculation unit 32 that performs interpolation calculation based on the result, a capture control unit 34 that controls capture of fisheye image data into the frame memory 31, a calculation unit 40 (portion surrounded by a broken line), A display memory 20 for storing the two-dimensional image data, a D / A converter 21 for converting the digital image data in the display memory 20 into analog image data, and an NTSC for converting the analog image data into an NTSC signal. And an encoder 22. The display memory 20, the D / A converter 21, and the NTSC encoder 22 constitute an image output unit, and are provided four each according to the number of cut-out image areas.

As described above, the frame memory 31 has a memory for two frames. While the original image data from the CCD camera 1 is fetched into one memory, coordinate conversion and interpolation calculation are performed from the other memory. The desired image data is cut out. The fisheye converter 10 is connected to a control personal computer 24 via an interface 23 (RS232C). The cutout area is set by the control personal computer 24.

Further, the calculation unit 40 includes a first coordinate calculation unit 35, a second coordinate calculation unit 36, a first lookup table 37 connected to the first coordinate calculation unit 35, and a second coordinate calculation unit 36. And a second lookup table 38 connected to the second lookup table 38. Explaining in relation to the coordinate conversion procedure described earlier, the first coordinate calculation unit 35 is a part that performs the calculation of the first step shown in FIG. 7, and has the (u, v) coordinates in the plane image. , The first projection coordinates (x ₂ , y ₂ , z ₂ ) on the hemisphere can be obtained. First look-up table 37 are tables for determining the distance L a correction coefficient k _1.

The second coordinate calculation unit 36 is a part that performs the calculation of the second step in FIG. 7, and the first projection coordinates (x ₂ , y ₂ , z ₂₎ obtained by the first coordinate calculation unit 35 ), The second projection coordinates (p ₁ , q ₁ ) on the fisheye imaging surface can be obtained. The second lookup table 38 is a table for obtaining the correction coefficient k _2.

<Description of Operation of Circuit> Next, the operation of the circuit shown in FIG. 8 will be described. Fish-eye image data captured by the CCD camera 1 is written to the frame memory 31 via the camera interface 30. The CCD camera 1 is based on the NTSC system, and the image data in the frame memory 31 is updated every 1/30 second. As is well known, in the NTSC system, interlaced scanning is performed, and an odd field (1/60 second) and an even field (1/60 second) are combined to form an image of one frame.

Since the coordinates (p, q) on the imaging surface corresponding to the coordinates (u, v) on the display screen are obtained by the calculation unit 40, the coordinates (p , Q) is read from the frame memory 31 and sent to the interpolation calculator 32.

The positions of the pixels on the fisheye image corresponding to the respective pixels of the planar image to be displayed on the monitor do not exactly match. In this case, a natural planar image can be obtained by reading several points of data of neighboring pixels from the frame memory 31, performing weighted averaging, and performing interpolation. The interpolated image data is stored in the order of # 1 → # 2 → # 3 → # 4 of the four display memories 20 corresponding to the image areas G1 to G4 to be cut out. The faster the time required to cut out the necessary image area from the frame memory 31, perform coordinate conversion and interpolation calculation, and store it in the display memory 20, the better. If the image data in the frame memory 31 is updated every 1/30 second, the hardware can cope with the above time at 1/30 second ÷ 4 = 1/120 second or more. Is preferred. The digital image data in the display memory 20 is converted into analog image data by a D / A converter 21, converted into an NTSC signal by an NTSC encoder 22, and output from an output terminal to the frame switcher device 11. Is done.

<Another Embodiment> Since the surveillance camera 1 according to the present embodiment is of the NTSC system, a versatile and inexpensive VTR can be used as the VTR in FIG. A monitoring camera of a progressive scanning method called a progressive scan camera may be used.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a monitoring system including an image conversion device.

FIG. 2 is a diagram showing a fisheye image.

FIG. 3 is a diagram showing a state recorded on a video tape.

FIG. 4 is a diagram showing a monitor display screen and the like.

FIG. 5 is a diagram showing a coordinate system setting;

FIG. 6 is a diagram to explain the correction coefficient k ₁

FIG. 7 is a view for explaining a coordinate conversion procedure by hardware.

FIG. 8 is a diagram showing a circuit block mounted on the fisheye converter.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 surveillance camera 2 fisheye lens 10 fisheye converter (image conversion device) 10 a cutout unit 10 b image conversion unit 10 c image output unit 11 frame switcher device 12 monitor 13 VTR

Claims (4)

[Utility model registration claims] 1. An image conversion apparatus for converting a fish-eye image captured using a fish-eye lens into a plane image for display, comprising: a cut-out unit that cuts out a predetermined number of image regions from the fish-eye image; An image conversion device, comprising: an image conversion unit that converts a fisheye image in an image region cut out by the method into a plane image; and an image output unit that outputs the converted plane image to a frame switcher device. 2. The image conversion apparatus according to claim 1, wherein an image area cut out by said cutout unit is configured to be changeable. 3. The image conversion apparatus according to claim 1, wherein the cutout section circulates and cuts the predetermined number of image areas at a predetermined cycle time. 4. An image conversion unit comprising: a first coordinate calculation unit for obtaining first projection coordinates of projecting the coordinates of the plane image onto a virtual object plane of a fisheye image plane; and the first coordinate calculation unit. 4. The apparatus according to claim 1, further comprising: a second coordinate calculation unit configured to calculate a second projection coordinate obtained by projecting the obtained first projection coordinate onto the fisheye image plane. 5. Image conversion device.