JP2004272578A - Image pickup unit and distortion correction method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image pickup unit capable of distortion correction for correcting the distortions caused in an image picked up in an image forming optical system having a negative distortion aberration at a speed higher than before, and a distortion correction method. <P>SOLUTION: In the image pickup unit which corrects the distortion so as to turn the image obtained by the image forming optical system having the distortion aberration which is larger in the peripheral area than in a center area, to a natural corrected image practically without the apparent distortion, the image of an optical image transmitted through the center area is enlarged or reduced and used for a part of the corrected image instead of correcting the distortion. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an imaging apparatus and a distortion correction method, and more particularly to an imaging apparatus that can execute a distortion correction process for correcting distortion generated in an image captured by an imaging optical system having negative distortion at a higher speed than before. And a distortion correction method.
[0002]
[Prior art]
When photographing a subject, there is a case where it is desired to photograph in a wide field of view. In particular, a surveillance camera desirably has a wide field of view due to the nature of surveillance. As one of the photographing methods satisfying such a demand, a method of photographing using a wide-distortion lens such as a fish-eye lens or a foveated lens can be considered.
[0003]
A wide-distortion lens is, for example, a lens that captures an angle of view of 180 degrees or more, so that a 180-degree field of view is contained in a finite screen as a completely similar image as in a normal wide-angle lens, It is impossible in principle. In order to fit a field of view with an angle of view of 180 degrees or more within a finite screen, the wide-distortion lens is given negative distortion, and all straight lines that do not pass through the center of the screen are curved in a barrel-shaped direction. It is reflected.
[0004]
FIG. 14 is a diagram illustrating a captured image captured by a wide-distortion lens and a distortion-corrected image obtained by correcting the distortion. FIG. 14A is a diagram illustrating a captured image captured by a wide distortion lens, and FIG. 14B is a diagram illustrating a distortion corrected image in which distortion has been corrected.
[0005]
In FIG. 14A, the captured image 1001 captured by the wide-distortion lens becomes a distorted image because all the straight lines that do not pass through the center of the captured image 1001 appear in a barrel-shaped curve as described above. It does not result in a natural image without similar distortion similar to a sight visible to the naked eye. For this reason, there is a problem that a viewer feels strange. Furthermore, since the distortion is large in the peripheral portion of the image, there is a visibility problem that it is difficult to determine what the subject is in the captured image 1001.
[0006]
Therefore, for a captured image captured by a fish-eye lens, each pixel of the captured image is rearranged, and a natural image in which the captured image is not substantially distorted when viewed, that is, a similar image similar to a sight visible to the naked eye. There is known a distortion correction process for obtaining a natural image having substantially no distortion (see, for example, Patent Document 1). For example, the captured image 1001 shown in FIG. 14A becomes a distortion-corrected image shown in FIG.
[0007]
In addition, the fisheye lens is a so-called circular image fisheye lens that completely captures a range of 180 degrees or more as a circular image on the imaging surface of the image sensor, and the 180 ° angle of view corresponds to a diagonal line on the imaging surface of the image sensor. There is a so-called diagonal fisheye lens that projects an image over the entire rectangular imaging surface.
[0008]
[Patent Document 1]
JP 2000-242773 A
[0009]
[Problems to be solved by the invention]
Incidentally, the distortion correction processing for correcting this distortion requires a huge amount of calculation. In particular, when the number of pixels is increased to obtain high resolution, the amount of calculation increases accordingly. Therefore, if the distortion correction processing is performed by dedicated hardware, the processing can be performed at high speed, but there is a problem that the cost is high.
[0010]
On the other hand, if the distortion correction processing is performed by a general LSI, for example, an ASIC (Application Specific Integrated Circuit) for a digital still camera, the distortion correction processing can be realized at low cost, but there is a problem that the processing speed is slow. In particular, in the case of a moving image, if the processing speed is reduced, the frame rate is reduced, so that a problem occurs in that the movement of the subject becomes awkward.
[0011]
The present invention has been made in view of the above problems, and has an image pickup apparatus capable of executing a distortion correction process at a higher speed than a conventional one when a distortion correction process is performed by a circuit having the same information processing capability. It is an object to provide a distortion correction method.
[0012]
[Means for Solving the Problems]
In order to achieve the above object, in the first means according to the present invention, an image obtained by an imaging optical system whose peripheral region has a large distortion with respect to a central region substantially reduces distortion when viewed. In an imaging apparatus that corrects distortion so as to obtain a natural corrected image that is not felt, an image of a light image transmitted through the central region is enlarged or reduced and used instead of correcting distortion in a part of the corrected image.
[0013]
The imaging apparatus according to the first means uses an optical system in which the peripheral region has large distortion with respect to the central region in the imaging optical system, so that not only an image with a wide field of view can be obtained but also the central region. The image of the light image transmitted through becomes a natural image that does not substantially feel distortion when viewed. For this reason, since the image of the optical image transmitted through the central region is enlarged or reduced, and the enlarged or reduced image is used as a part of the corrected image, the distortion correction processing for correcting the distortion of the portion is not required. Thus, the distortion correction processing can be executed at high speed. In particular, when the corrected image itself is an image obtained by enlarging or reducing the image of the light image transmitted through the central region, it is not necessary to execute the distortion correction processing itself, so that a corrected image can be obtained at higher speed.
[0014]
In the second means according to the present invention, an image obtained by an imaging optical system having a large distortion in a peripheral region with respect to a central region is a natural corrected image which does not substantially feel distortion when viewed. When correcting the distortion of the image, the image capturing apparatus corrects the distortion at a fixed pixel interval for each pixel of the image to obtain a corrected image.
[0015]
The image pickup apparatus according to the second means performs the distortion correction processing at a fixed pixel interval instead of performing the distortion correction processing for all the pixels of the image. Correction processing can be performed.
[0016]
In a third aspect according to the present invention, in the imaging device according to the second aspect, for each pixel within a certain range of the image, distortion is corrected for all pixels.
[0017]
The imaging device according to the third means corrects the distortion of all the pixels within a certain range, so that the image quality can be improved in that range.
[0018]
In a fourth aspect according to the present invention, in the imaging device according to the second aspect, the image of the light image transmitted through the central region is enlarged or reduced and used instead of an image within a certain range of the corrected image.
[0019]
The imaging apparatus according to the fourth means uses the image of the light image transmitted through the central region in an enlarged or reduced manner instead of an image within a certain range of the corrected image, and thus can improve the image quality in that range. it can.
[0020]
In the fifth means according to the present invention, an image obtained by an imaging optical system in which a central region has a large distortion with respect to a central region is a natural corrected image in which a user does not substantially feel distortion when viewed. A distortion correction method for correcting distortion in an image includes a step of enlarging or reducing an image formed by an optical image transmitted through the central area, and a step of correcting distortion of an image formed by an optical image transmitted through the peripheral area.
[0021]
The distortion correction method according to the fifth means uses an optical system in which the peripheral region has a large distortion with respect to the central region in the imaging optical system, so that not only an image with a wide field of view can be obtained but also the central region. The image of the light image transmitted through the region becomes a natural image that does not substantially feel distortion when viewed. For this reason, since the image of the optical image transmitted through the central region is enlarged or reduced, and the enlarged or reduced image is used as a part of the corrected image, the distortion correction processing for correcting the distortion of the portion is not required. Thus, the distortion correction processing can be executed at high speed.
[0022]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments according to the present invention will be described with reference to the drawings. In each drawing, the same components are denoted by the same reference numerals, and description thereof will be omitted. First, the configuration of the first embodiment will be described.
(Configuration of First Embodiment)
The first embodiment is a case where the imaging device according to the present invention is applied to a monitoring camera as an example.
[0023]
FIG. 1 is a diagram illustrating a configuration of a monitoring system according to the first embodiment. In FIG. 1, a monitoring system 1 includes a monitoring camera 2 that captures a predetermined monitoring target area, and a controller 3 such as a personal computer (PC) or a mobile phone. It is connected via a communication network 4.
[0024]
In the monitoring system 1, when the monitoring camera 2 captures an image of a monitoring target, the monitoring camera 2 transmits image data of the image to the controller 3 via the communication network 4. On the other hand, when the user inputs an operation command such as an instruction to change the photographing direction to the monitoring camera 2, the controller 3 transmits the operation command to the monitoring camera 2 via the communication network 4. Then, the monitoring camera 2 operates based on the operation command. Thus, the image captured by the monitoring camera 2 is displayed on the display unit of the controller 3, and the operation of the monitoring camera 2 can be remotely operated.
[0025]
The communication network 4 may be a communication network constructed by any wireless or wired transmission medium, and may be any communication standard. For example, various communication protocols such as Ethernet (trademark) (including wireless LAN) and TCP / IP can be used. Further, the surveillance camera 2 and the controller 3 may be in a form of directly communicating with each other by a wireless or wired cable based on a communication standard such as IrDA or Bluetooth (trademark).
[0026]
FIG. 2 is a diagram showing a configuration of a monitoring camera used in the monitoring system. In FIG. 2, the monitoring camera 2 includes a camera 21, a frame 22 bent in a U-shape, and a geared motor 23 for moving the optical axis L of the camera 21 (corresponding to a shooting direction and a monitoring direction) in a vertical direction (tilt direction). And a geared motor 24 for moving the optical axis L of the camera 21 in the left-right direction (pan direction).
[0027]
The camera 21 is attached to the frame 22 with the tilt-direction rotating shafts 25 projecting from the left and right side surfaces penetrating through holes 22B formed in the side surfaces 22A and 22A 'of the frame 22. The output shaft of the geared motor 23 is attached to the tip of the rotating shaft 25 protruding through the side surface 22A. At the center of the lower surface of the frame 22, a pan-direction rotating shaft 26 protrudes downward, and an output shaft of the geared motor 24 is attached to a tip of the rotating shaft 26. The housing of the geared motor 23 is fixed to the frame 22 by a fixing member (not shown), and is configured to be rotatable integrally with the frame 22 in the tilt direction. The geared motor 24 is fixed to a support 27 of the monitoring camera 2.
[0028]
In such a configuration, when the geared motor 24 is driven to rotate, the frame 22 rotates around the rotation axis 26, whereby the optical axis L of the camera 21 moves in the pan direction. On the other hand, when the geared motor 23 is driven to rotate, the camera 21 rotates around the rotation axis 25, whereby the optical axis L of the camera 21 moves in the tilt direction.
[0029]
In the following description, the operation of the monitoring camera 2 in which the optical axis L of the camera 21 moves in the pan direction is referred to as a pan operation, and the operation of the monitoring camera 2 moving in the tilt direction is referred to as a tilt operation.
[0030]
FIG. 3 is a block diagram illustrating a configuration of the monitoring camera. 3, the monitoring camera 2 includes an imaging unit 30, an image generation unit 31, an image data buffer 32, an image processing unit 33, a driving unit 34, a control unit 35, an image data recording unit 36, a storage unit 37, and a communication I / F. And an I / F unit 39.
[0031]
The imaging unit 30 includes an imaging optical system 301 and an imaging element 302. Light from the subject is formed on the imaging surface of the image sensor 302 by the imaging optical system 301, and becomes an optical image of the subject.
[0032]
The imaging element 302 is a photoelectric conversion element in which a plurality of light receiving elements for converting received light into an electric signal according to the light intensity are two-dimensionally arranged in, for example, a honeycomb shape or a matrix shape. This is a CCD color area sensor in which R (red), G (green), and B (blue) color filters are arranged at a ratio of 1: 2: 1 on the imaging surface of the element. The image sensor 302 converts the light image of the subject formed by the imaging optical system 301 into electrical signals (image signals) of R (red), G (green), and B (blue) color components. The image signal is output to the image generation unit 31 as an image signal of each of G and B colors. The imaging device 302 controls the imaging operation such as the start and end of the exposure operation and the reading (horizontal synchronization, vertical synchronization, transfer) of the output signal of each pixel in the imaging device 302 under the control of the control unit 35.
[0033]
Note that the imaging device 302 is not limited to the above-described CCD but may be a CMOS solid-state imaging device, and may be a monochrome imaging device without being limited to a color imaging device.
[0034]
The imaging optical system 301 is a lens that forms an optical image of a subject on an imaging surface of the imaging element 302. In a region (center region) within a certain range around the optical axis, the value of the distortion is equal to or less than a predetermined value. In a region around the central region (peripheral region), the value of the distortion is larger than a predetermined value. In particular, the imaging optical system 301 of the present embodiment is an optical system in which the value of the distortion increases in the peripheral region as it goes to the peripheral portion.
[0035]
FIG. 4 is a diagram illustrating characteristics of the imaging optical system according to the first embodiment. FIG. 4A is a graph showing the distortion / angle-of-view characteristics, in which the horizontal axis is the distortion in percentage and the vertical axis is the angle of view θ in degrees (°). FIG. 4B is a graph showing the angle-of-view and image-height characteristics, in which the horizontal axis is the angle of view θ and the horizontal axis is the image height h.
[0036]
As shown in FIG. 4A, the distortion / field angle characteristic, which is the relationship between the distortion S and the angle of view θ, is such that when the angle of view θ is equal to or smaller than a predetermined value θi, When the angle of view θ exceeds a predetermined value θi, the distortion S sharply increases. Here, the predetermined value Si in the distortion S is a value that looks similar to a natural image with substantially no distortion when an image of the subject light image transmitted through the central region of the imaging optical system 301 is viewed by a human. For example, Si = about 3% (in this case, θi = about 8 °). Needless to say, even if the predetermined value Si is set to a value of 3% or less, for example, about 2% or about 1%, a natural image is obtained in which distortion is not substantially felt when viewed.
[0037]
FIG. 4A shows the characteristics of the imaging optical system 301 in which the distortion is about -70% at a half angle of view of about 50 degrees.
[0038]
Due to this characteristic, the angle of view / image which is a relationship between the angle of view θ and the height (hereinafter, referred to as “image height”) h of the optical image formed on the imaging surface of the image sensor 302 by the imaging optical system 301. The high characteristic is a curve shown in FIG. That is, in an area where the angle of view θ is small (the area on the left side of the dotted line shown in FIG. 4B, corresponding to the central area of the imaging optical system 301), the image height h is substantially linear with respect to the angle of view θ. Therefore, the amount of change with respect to the unit change of the angle of view θ is large. On the other hand, in an area where the angle of view θ is large (an area on the right side of the dotted line shown in FIG. 4B, corresponding to the peripheral area of the imaging optical system 301), the image height h is nonlinear with respect to the angle of view θ. , The amount of change with respect to the unit change of the angle of view θ gradually decreases as the angle of view θ increases, and the image height h saturates to a substantially constant value. That is, the resolution is high in an area where the angle of view θ is small, and the resolution is low but the amount of information is large in a peripheral area where the angle of view θ is large.
[0039]
One of the imaging optical systems 301 having such characteristics is a lens called a foveated lens. The fovea lens is a lens having a function of enlarging an image in a region of interest (corresponding to the central region described above) in a visual field and compressing the image in a peripheral region thereof (corresponding to the peripheral region described above). There are features such as high resolution in the region and a natural image with less noticeable distortion in the region of interest.
[0040]
Since the surveillance camera 2 according to the first embodiment uses the imaging optical system 301 having the characteristics shown in FIG. 4, the central part (image central area X) of the captured image has passed through the central area of the imaging optical system 301. Since the image is based on the subject light image, a natural image with high resolution and less noticeable distortion can be obtained, and a wide field of view can be obtained.
[0041]
Returning to FIG. 3, the image generation unit 31 performs well-known image processing such as determination of an appropriate black level, γ correction, white balance adjustment (WB adjustment), contour correction, and color unevenness correction on the entire image. , Image data of each pixel is generated from the image signal. The image data generated by the image generation unit 31 is output to the image data buffer 32.
[0042]
The image data is configured by associating, for each pixel, position data representing the position of the pixel with luminance data representing the luminance of the pixel. The luminance data is composed of the luminance value of each pixel when the surveillance camera 2 is to photograph in monochrome, and the luminance value of the red component in each pixel when the surveillance camera 2 is to photograph in color. It is composed of a luminance value of a green component and a luminance value of a blue component, or a luminance value and a value of each color difference.
[0043]
The image data buffer 32 is a memory that temporarily stores image data and is used as a work area for performing a process described later on the image data by the image processing unit 33, and is, for example, a RAM.
[0044]
The image processing unit 33 performs, for the image data in the image data buffer 32, a distortion correction process for correcting an image distorted due to distortion into a natural image similar to a sight similar to a sight visible to the naked eye, and substantially without distortion. Is a circuit that performs image processing. The drive unit 34 includes the geared motors 23 and 24, and drives the optical axis L of the camera 21 (monitoring camera 2) in the pan direction and the tilt direction according to a control signal output from the control unit 35.
[0045]
The image data recording unit 36 is a circuit that records the image data stored in the image data buffer 32 on a recording medium. The recording medium is, for example, a CD-R, a hard disk, a memory card, or the like. The storage unit 37 is a storage circuit that stores various programs necessary for the operation of the monitoring camera 2, data required during the execution of the program, and data generated during the execution of the program, and includes, for example, a ROM and a RAM. Is done.
[0046]
The communication I / F unit 38 is an interface for performing communication with the communication network 4, and transmits a communication packet containing image data to the controller 3 and a communication packet containing an operation command from the controller 3. And the like. The I / F unit 39 is an interface for transmitting and receiving image data to and from an external device. For example, the I / F unit 39 is an interface that conforms to a standard such as USB or IEEE1394.
[0047]
The control unit 35 includes, for example, a microprocessor or the like, and includes an imaging unit 30, an image generation unit 31, an image data buffer 32, an image processing unit 32, a driving unit 34, an image data recording unit 36, a storage unit 37, and a communication unit. It controls each unit of the I / F unit 38 and the I / F unit 39.
[0048]
FIG. 5 is a block diagram showing the configuration of the controller. 5, the controller 3 includes an operation unit 41, a display unit 42, a control unit 43, and a communication I / F unit 44.
[0049]
The operation unit 41 is a device that inputs operation commands for instructing various operations on the monitoring camera 2 such as a pan operation, a tilt operation, recording of image data, a request for transmitting image data, and the like. For example, when the controller 3 is a PC Corresponds to a keyboard or a mouse, and in the case of a mobile phone, corresponds to a numeric keypad or a cursor key.
[0050]
The display unit 42 is a display that displays an image related to the image data transmitted from the monitoring camera 2 via the communication network 4. When the controller 3 is a PC, the display unit 42 corresponds to a monitor such as a CRT display or an LCD. In the case of a mobile phone, it corresponds to an LCD display unit.
[0051]
The communication I / F unit 44 is an interface for performing communication with the communication network 4, and receives a communication packet containing image data from the monitoring camera 2 and a communication packet containing an operation command to the monitoring camera 2. Is transmitted.
[0052]
The control unit 43 includes, for example, a microcomputer including a ROM that stores a control program for controlling the controller 3 and a RAM that temporarily stores data, and includes an operation unit 41, a display unit 42, and a communication I / F unit 44. The operation of the controller 3 is controlled by organic control.
[0053]
Next, the operation of the first embodiment will be described.
(Operation of First Embodiment)
The surveillance camera 2 in the first embodiment performs the distortion correction process only in the peripheral region by using the image of the central region without performing the distortion correction, utilizing the characteristics of the imaging optical system 301 in the distortion correction process. It is.
[0054]
FIG. 6 is a diagram illustrating the operation of the surveillance camera according to the first embodiment. FIG. 7 is a diagram showing a display screen displayed on the display unit of the controller. FIG. 8 is a diagram for explaining the distortion correction processing. FIG. 9 is a diagram illustrating an image after the distortion correction processing in the telephoto case according to the first embodiment. FIG. 10 is a diagram illustrating an image after distortion correction processing in the case of a wide angle in the first embodiment.
[0055]
In FIG. 6, when the control unit 35 of the monitoring camera 2 receives an operation command of an instruction to transmit a monitoring target image from the controller 3 via the communication I / F unit 38, the control unit 35 captures an image of the monitoring target with the imaging unit 30, The image generating unit 31 generates image data based on an image signal from the image sensor 302, and stores the generated image data in the image data buffer 32. An image based on image data obtained by the imaging optical system 301 having the characteristics shown in FIG. 4 is referred to as a wide distortion image. The control unit 35 transmits the image data of the wide distortion image to the controller 3 via the communication I / F unit 38 and the communication network 4 (S11).
[0056]
Upon receiving the image data via the communication network 4 and the communication I / F unit 44, the controller 3 displays the image data on the display unit 42. Note that communication between the control unit 35 of the monitoring camera 2 and the control unit 43 of the controller 3 is performed via the communication I / F unit 38, the communication network 4, and the communication I / F unit 44. In the following description, this description is omitted.
[0057]
The display screen of the display unit 42 of the controller 3 includes, for example, as shown in FIG. 7, an image display area 101 which is an area for displaying an image generated by image data from the monitoring camera 2, and image data of a wide-angle image. For inputting an operation command instructing the surveillance camera 2 to transmit an image, and an operation command instructing the surveillance camera 2 to transmit image data of a telephoto image. And a "image recording" 104 which is a button for inputting an operation command for instructing the monitoring camera 2 to record image data on the recording medium in the image data recording unit 36. Is done. The “image recording” 104 not only instructs the surveillance camera 2 to record the image data on the recording medium in the image data recording unit 36 but also includes an image data recording unit (not shown) provided in the controller 3. May include an instruction to store the image data of the image displayed in the image display area 101. For the “wide angle” 102, the “telephoto” 103, and the “image recording” 104, for example, in the case of a PC, an operation command corresponding to each button is input by moving a cursor to a desired button with a mouse and clicking. .
[0058]
First, a wide distortion image is displayed in the image display area 101 of the display unit 41 of the controller 3 as shown in FIG.
[0059]
The user inputs the angle of view of the display image to be displayed in the image display area 101 to the controller 3 using “wide angle” 102 or “telephoto” 103. The control unit 43 of the controller 3 transmits the angle of view input by the operation unit 41 and the display unit 42 to the control unit 35 of the monitoring camera 2.
[0060]
The control unit 35 of the monitoring camera 2 receives the angle of view of the display image to be displayed (S12). The control unit 35 determines whether the received angle of view is a wide angle or a telephoto (S13). As a result of the determination, the control unit 35 performs the process of S14 when it is telephoto (“telephoto” of S13), and performs the process of S21 when it is wide-angle (“wide-angle” of S13).
[0061]
In S <b> 14, the control unit 35 causes the image processing unit 33 to perform size conversion to enlarge the size of the image central area X of the wide-distorted image to the size of the image display area 101.
[0062]
That is, in FIG. 8, the size of the wide-distorted image 111 is M pixels (pix) in the horizontal direction and N (pix) in the vertical direction, and is a region corresponding to the image of the subject light image transmitted through the central region of the imaging optical system 301. The size of a certain image central region X is defined as a horizontal direction A (pix) (A ≦ M) and a vertical direction B (pix) (B ≦ N). In FIG. 9, the size of the image display area 101 is defined as a horizontal direction K (pix) (K ≦ M) and a vertical direction L (pix) (L ≦ N). In order to easily perform the size conversion process, the ratio of the number of horizontal pixels to the number of vertical pixels of the wide-distorted image 111, the ratio of the number of horizontal pixels to the number of vertical pixels of the image central region X, and the image display The ratio of the number of pixels in the horizontal direction to the number of pixels in the vertical direction in the area 101 is all the same (M: N = A: B = K: L).
[0063]
As described above, the central region of the imaging optical system 301 is set to a range in which an image of a subject light image transmitted through this region can be viewed as a natural image having substantially similar distortion when viewed by a human. The image in the image central region X is a natural image that does not substantially feel distortion when viewed.
[0064]
When the size of the image display area 101 and the size of the image central area X are set as described above, the image processing unit 33 performs image data corresponding to the image of the image central area X stored in the image data buffer 32. Is calculated by a known method so that the image in the image central region X is multiplied by K / A. That is, in the image data, the position after the size change is calculated, and the luminance data of the image data is assigned to the calculated position. By multiplying the image in the image central area X by K / A, the image in the image central area X is enlarged to the size of the image display area 101. This size conversion can be performed at high speed by using a size conversion function in an ASIC for a digital still camera.
[0065]
Then, the control unit 35 transmits the image data of the image in the image central area X multiplied by K / A to the controller 3 (S15).
[0066]
When receiving the image data, the control unit 43 of the controller 3 displays the image data on the display unit 42. For example, when the area indicated by the broken line in FIG. 8 is the image center area X, the image data of this house is multiplied by K / A to obtain the image shown in FIG. You.
[0067]
As described above, since the distortion correction processing on the telephoto display image merely enlarges the image of the central image area X, it is not necessary to execute a normal distortion correction processing described later, and the distortion correction processing is performed at a higher speed than in the related art. Can be performed.
[0068]
On the other hand, in S21, the image processing unit 33 calculates the angle of view θx of the central image area X of the wide distortion image (S21). The angle of view θx can be obtained from Expression 1 assuming that a function indicating the relationship between the image height h and the angle of view θ is given as θ = g (h).
θx = g (A / 2) × 2 (1)
Here, as the function g, for example, the characteristic curve shown in FIG. 4B may be created by actually measuring the angle-of-view / image height characteristics of the imaging optical system 301 to obtain the function g.
[0069]
Next, the image processing unit 33 calculates the size of the pasting area Y that is an area for displaying an image based on the central image area X of the wide-distortion image in the wide-angle display image (S22). Assuming that the size of the pasting area Y is C (pix) (C ≦ K) in the horizontal direction and D (pix) (D ≦ L) in the vertical direction, and the angle of view of the image display area 101 is θv, the size of the pasting area Y is It can be obtained from Equations 2 and 3.
C = K × tan (θx / 2) / tan (θv / 2) (2)
D = C × B / A (3)
Note that the ratio of the number of horizontal pixels to the number of vertical pixels in the pasting area Y is equal to the ratio of the number of horizontal pixels to the number of vertical pixels in the image display area 101 (C: D = K: L).
[0070]
Next, the image processing unit 33 converts the image data corresponding to the image in the image central area X stored in the image data buffer 32 into a known method so that the image in the image central area X is multiplied by C / A. Is used to calculate the image data (S23). By multiplying the image of the image central region X by C / A, the image of the image central region X is enlarged or reduced to the size of the pasting region Y.
[0071]
Next, the image processing unit 33 converts the image data of each pixel in the image of the area Z other than the pasting area Y in the image display area 101 into the image data of each pixel in the image other than the central image area X in the wide-distortion image 111. It is generated by performing normal distortion correction processing on the other hand (S24).
[0072]
Hereinafter, a calculation method of the normal distortion correction processing will be described. Here, the image data of the wide distortion image is referred to as original image data, and the pixel data of each pixel constituting the original image data is referred to as original pixel data. An arbitrary pixel in the distortion-corrected image obtained by correcting the distortion of the wide-distortion image is referred to as a target pixel B, and its coordinates are set to B (i, j). In the distortion correction processing, the relationship between the coordinates B (i, j) of the target pixel B and the coordinates Q (u, v) of the pixel Q of the original image corresponding to the target pixel B may be obtained. By obtaining this relationship, the luminance data of the original image data is allocated to the position after the calculation, and the image data of the distortion corrected image can be obtained.
[0073]
First, in the distortion-corrected image, the distance d (dx: x component, dy: y component) from the center A (K / 2, L / 2) of this image to the pixel of interest B is calculated by Expressions 7 to 7 using Expressions 4 to 6. It becomes.
dx = (K / 2−i) (4)
dy = (L / 2−j) (5)
d = √ (dx ² + Dy ² ) ・ ・ ・ (6)
By
d = √ ｛(K / 2-i) ² + (L / 2-j) ² ・ ・ ・ ... (7)
It becomes.
[0074]
Further, when it is assumed that the relationship between the image height Y, the focal length f, and the angle of view Θ is taken using a normal lens represented by Y = f · tan Θ, the distortion-corrected image is taken. The incident angle φ of the light converted to the pixel data of the pixel located at the coordinate B (i, j) is the incident angle of the light converted to the pixel data of the pixel located at the coordinate Q (u, v) in the original image. Is the same as
[0075]
Therefore, if the horizontal angle of view of the distortion-corrected image is θ (rad), the incident angle φ of the light converted into the pixel data of the pixel located at the coordinate B (i, j) in the distortion-corrected image is expressed by the following equation (8). And Equation 9 hold, and Equation 10 results.
X = (K / 2) / tan (θ / 2) (8)
tanφ = d / X (9)
φ = tan ^-1 {D × tan (θ / 2) / (K / 2)} (10)
Then, assuming that the distance (image height) from the center P (M / 2, N / 2) to the coordinates Q (u, v) in the original image is h, the distance h is the incident light obtained by Expression (10). It is expressed as a function h = f (φ) with the angle φ as a parameter. The function f is obtained according to the radius of curvature of the imaging optical system 301, but may be obtained by actual measurement similarly to the function g.
[0076]
On the other hand, since Expressions 11 and 12 hold, Expressions 13 and 14 are obtained.
h: d = (u−M / 2): dx (11)
h: d = (v−N / 2): dy (12)
u = M / 2 + h × (dx / d) (13)
v = N / 2 + h × (dy / d) (14)
Therefore, the coordinates Q (u, v) of the pixel data in the original image corresponding to the pixel data located at the coordinates B (i, j) can be obtained from h = f (φ) and Expressions 13 and 14. it can.
[0077]
Returning to FIG. 6, the image processing unit 33 displays the image data of the pasting area Y generated in S23 at the center of the image display area 101, and the image data of the area Z other than the pasting area Y generated in S24. Are transmitted to the controller 3 so as to be displayed (S25). Note that both image data may be combined with the image data of one image, and the combined image data may be transmitted to the controller 3.
[0078]
When receiving both the image data, the controller 3 first displays the image data of the pasting area Y in the center of the image display area 101, and displays the image data of the area Z other than the pasting area Y around it. Thus, the wide-angle image of the monitoring target is displayed on the display unit 42 of the controller 3. For example, the wide distortion image shown in FIG. 8 is displayed in the image display area 101 of the display unit 42 of the controller 3 as shown in FIG. The missing area 121 in FIG. 10 is generated because there is a light image that is not received by the image sensor 302 among the circular images of the wide distortion image.
[0079]
When recording a display image, the user inputs an operation command for recording a display image to the controller 3 by “image recording” 104. The controller 3 stores the image data in a storage unit (not shown in FIG. 5) and transmits the operation command to the monitoring camera 2. The surveillance camera 2 receiving the operation command records the image data in the image data recording unit 36.
[0080]
In the first embodiment, when a distortion-corrected image in telephoto is displayed in the image display area 101, the distortion-corrected image to be displayed is generated by enlarging or reducing the image in the central image area X of the wide distortion image 111. Therefore, there is no need to perform a normal distortion correction process. For this reason, it is possible to generate and display a distortion-corrected image in telephoto at a higher speed than in the related art.
[0081]
When a wide-angle distortion-corrected image is displayed in the image display area 101, a part of the distortion-corrected image to be displayed is generated by enlarging or reducing the image in the central image area X of the wide-distorted image 111. Then, an image of the remaining portion is generated by performing a normal distortion correction process. For this reason, it is not necessary to generate all of the distortion-corrected images by normal distortion correction processing, so that a distortion-corrected image at a wide angle can be generated and displayed at a higher speed than in the related art. In the central image area X, a natural image that does not substantially distort the appearance is obtained due to the characteristics of the imaging optical system 301. Therefore, it is necessary to maintain the central image quality of the image that is easily noticed in the image. Can be.
[0082]
Therefore, since the distortion correction processing can be performed at high speed, an inexpensive integrated circuit such as an LSI for general use or an ASIC for digital still camera can be used, and the cost of the monitoring camera 2 can be reduced. it can.
[0083]
Also, when the surveillance camera 2 transmits a moving image, the distortion correction processing can be executed at a high speed. Therefore, when the distortion correction processing is performed by a circuit having the same processing capability, the frame rate is improved as compared with the related art. Therefore, a moving image having a smoother motion can be obtained.
[0084]
Next, a second embodiment will be described.
(Configuration of the second embodiment)
In the first embodiment, the telephoto image is generated by enlarging the image of the central image area X in the wide distortion image to the size of the image display area 101, and the wide angle image is generated by the central image area X in the wide distortion image. Is enlarged or reduced to the size of the pasting region Y, and the image of the region Z other than the pasting region Y is generated by performing the distortion correction process on the wide-distorted image 111. In the second embodiment, the telephoto image is generated. Is generated by enlarging the image of the central image area X in the wide-distortion image to the size of the image display area 101. The wide-angle image executes the normal distortion correction processing for all the pixels in the pasting area Y, An area Z other than the pasting area Y is generated by executing a normal distortion correction process at a ratio of one pixel to s pixels. Note that a region corresponding to the pasting region Y in the first embodiment is referred to as a distortion correction central region W in the second embodiment. The size of the distortion correction central region W is E (pix) × F (pix), and in this embodiment, E = C and F = D.
[0085]
Therefore, the configuration of the monitoring system 1, the configuration of the monitoring camera 2, and the configuration of the controller 3 are the same as those in the first embodiment, except that the processing of the image processing unit 33 of the monitoring camera 2 is performed as described later. Therefore, the description is omitted.
[0086]
Next, the operation of the second embodiment will be described.
(Operation of Second Embodiment)
The operation of the surveillance camera according to the second embodiment is the same as that of the first embodiment except for a process of generating image data of a wide-angle image. That is, since the processing of S11 to S15 in FIG. 6 in the first embodiment is the operation of the surveillance camera in the second embodiment as it is, a description thereof will be omitted, and the processing of generating image data of a wide-angle image will be omitted. explain.
[0087]
FIG. 11 is a diagram illustrating the operation of the surveillance camera when a wide-angle image is generated. FIG. 12 is a diagram illustrating an image after distortion correction processing in the case of a wide angle in the second embodiment.
[0088]
In FIG. 11, the image processing unit 33 generates image data of a 1 / s reduced wide-distorted image obtained by reducing the wide-distorted image to 1 / s based on the image data stored in the image data buffer 32 (S31). ).
[0089]
Next, the image processing unit 33 performs the above-described normal distortion correction processing on the 1 / s reduced wide distortion image (S32). Since the normal distortion correction processing is performed on the 1 / s reduced engineering distortion image reduced to 1 / s, the normal distortion correction processing is executed at a ratio of one pixel to s pixels. For this reason, since the distortion correction processing is performed at a fixed pixel interval s for each pixel in the wide distortion image, the number of pixels to be processed is reduced as compared with the case where the distortion correction processing is performed on the wide distortion image. As a result, the processing time of the distortion correction processing is reduced. This s is a value determined by the image quality or the like required for the image obtained after the distortion correction. In the present embodiment, s = 2.
[0090]
Next, the image processing unit 33 performs an enlargement process of enlarging the 1 / s reduced wide-distorted image after the distortion correction process by s times. As a result, the size becomes the size when the wide distortion image is directly subjected to distortion correction processing. Here, since the distortion correction processing is performed at a rate of one pixel for s pixels, some pixels do not have luminance data. For this reason, the image processing unit 33 generates the image data for the pixel for which the distortion correction process has not been performed by interpolating based on the image data of the adjacent pixel (S33).
[0091]
In this interpolation, for example, the luminance data of a pixel adjacent to one of the pixels is used as it is as the luminance data of the pixel. Further, for example, a linear function indicating the relationship between the position and the luminance is obtained using the luminance data of both pixels adjacent to the pixel, and the luminance data of the pixel is determined from the linear function (linear interpolation). Further, for example, the luminance data of the pixel is obtained by averaging the luminance data of the peripheral pixels of the pixel.
[0092]
Next, the image processing unit 33 performs a normal distortion correction process on the image in the image central region X of the wide distortion image, and generates image data after the distortion correction process (S34).
[0093]
Next, the image processing unit 33 transmits both the image data to the controller 3 so that the image generated in S33 is displayed first and the image data generated in S34 is displayed in a superimposed manner at the center of the image display area 101. (S35). Note that both image data may be combined with the image data of one image, and the combined image data may be transmitted to the controller 3.
[0094]
Upon receiving both image data, the controller 3 first displays the image data generated in S33 on the entire image display area 101, and displays the image data generated in S34 in the distortion correction central area W substantially at the center of the image display area 101. Is displayed over the display. Thus, the wide-angle image of the monitoring target is displayed on the display unit 42 of the controller 3. For example, the wide distortion image shown in FIG. 8 is displayed in the image display area 101 of the display unit 42 of the controller 3 as shown in FIG.
[0095]
In the second embodiment, when displaying a wide-angle distortion-corrected image in the image display area 101, normal distortion correction processing is performed on each pixel in the wide-distortion image at a fixed pixel interval, and the image display area 101 is displayed. In the distortion correction central region W, which is approximately the center of the wide distortion image, normal distortion correction processing is performed on all pixels in the wide distortion image. For this reason, it is not necessary to generate all of the distortion-corrected images by normal distortion correction processing, so that a distortion-corrected image at a wide angle can be generated and displayed at a higher speed than in the related art. Then, in the distortion correction central area W, the normal distortion correction processing is performed on all the pixels in the wide distortion image, so that the image quality at the center of the image that is easily noticed in the image can be maintained.
[0096]
Next, a third embodiment will be described.
(Configuration of Third Embodiment)
In the third embodiment, the image central region X of the wide-distorted image is replaced with the image central region X of the wide-distorted image in the same manner as in the first embodiment, instead of performing the distortion correction process on the image of the image central region X of the wide-distorted image in S34 in the second embodiment. This is an embodiment in which an enlarged or reduced image is pasted.
[0097]
Therefore, the configuration of the monitoring system 1, the configuration of the monitoring camera 2, and the configuration of the controller 3 are the same as those in the first embodiment, except that the processing of the image processing unit 33 of the monitoring camera 2 is performed as described later. Therefore, the description is omitted.
[0098]
Next, the operation of the third embodiment will be described.
(Operation of Third Embodiment)
The operation of the surveillance camera according to the third embodiment is the same as that of the first embodiment except for a process of generating image data of a wide-angle image. That is, since the processing of S11 to S15 in FIG. 6 in the first embodiment is the operation of the surveillance camera in the second embodiment as it is, a description thereof will be omitted, and the processing of generating image data of a wide-angle image will be omitted. explain.
[0099]
FIG. 13 is a diagram illustrating an operation of the surveillance camera when a wide-angle image is generated according to the third embodiment.
[0100]
In FIG. 13, the image processing unit 33 generates image data of a 1 / s reduced wide-distorted image obtained by reducing the wide-distorted image to 1 / s based on the image data stored in the image data buffer 32 (S41). ). Next, the image processing unit 33 performs a normal distortion correction process on the 1 / s reduced wide distortion image (S42). Next, the image processing unit 33 performs an enlargement process of enlarging the 1 / s reduced wide-distorted image after the distortion correction process by s times. Then, the image processing unit 33 generates the image data for the pixels for which the distortion correction processing has not been performed by interpolating based on the image data of the adjacent pixels (S43).
[0101]
Next, the image processing unit 33 calculates the angle of view θx of the central image area X of the wide-distortion image by using Expression 1 (S44). Next, the image processing unit 33 calculates the size of the pasting area Y, which is an area for displaying an image based on the central image area X of the wide-distortion image in the wide-angle display image, using Expressions 2 and 3 (S45). Next, the image processing unit 33 converts the image data corresponding to the image in the image central area X stored in the image data buffer 32 into a known method so that the image in the image central area X is multiplied by C / A. Is used to calculate the image data (S46).
[0102]
The image processing unit 33 transmits both image data to the controller 3 so that the image generated in S43 is displayed first and the image data generated in S46 is displayed in a superimposed manner at the center of the image display area 101 (S47). ). Note that both image data may be combined with the image data of one image, and the combined image data may be transmitted to the controller 3.
[0103]
Upon receiving both image data, the controller 3 first displays the image data generated in S43 on the entire image display area 101, and superimposes the image data generated in S47 on the paste area Y substantially at the center of the image display area 101. To display. Thus, the wide-angle image of the monitoring target is displayed on the display unit 42 of the controller 3.
[0104]
In the third embodiment, when a wide-angle distortion-corrected image is displayed in the image display area 101, normal distortion correction processing is performed on each pixel in the wide-distortion image at a fixed pixel interval, and the image display area 101 is displayed. Is generated by enlarging or reducing the image in the central image area X of the wide-distortion image 111. For this reason, since it is not necessary to generate all of the distortion-corrected images by normal distortion correction processing, a wide-angle distortion-corrected image can be generated and displayed at a higher speed than in the related art. In the central image area X, a natural image that does not substantially distort the appearance is obtained due to the characteristics of the imaging optical system 301. Can be.
[0105]
In the first to third embodiments, the embodiment in which the surveillance camera 2 as the imaging device and the controller 3 for operating the same are separately described, but the imaging device and the controller 3 are integrally configured. May be.
[0106]
The main inventions disclosed in the present specification are summarized below.
(Supplementary Note 1) Imaging that corrects distortion so that an image obtained by an imaging optical system whose peripheral region has a large distortion with respect to a central region becomes a natural corrected image that does not substantially feel distortion when viewed. An imaging apparatus, wherein an image of a light image transmitted through the central region is enlarged or reduced and used instead of correcting distortion in a part of the corrected image.
(Supplementary Note 2) An image forming optical system having a large distortion in a peripheral region with respect to a central region, and an image formed by the image forming optical system on an imaging surface in which a plurality of light receiving elements are two-dimensionally arranged. An image sensor that photoelectrically converts an optical image with the plurality of light receiving elements and outputs an image signal; an image generating unit that generates image data of each pixel in the image of the optical image based on the image signal; An image in which the image data of each pixel in the image of the light image is distortion-corrected and the image data of each pixel in the corrected image is calculated so that the image of FIG. An image processing unit comprising: a processor configured to correct image data of each pixel in an image of the light image transmitted through the central region into a part of image data of each pixel in the corrected image. Do An imaging device characterized in that the imaging device is used without being used.
(Supplementary Note 3) Imaging that corrects distortion so that an image obtained by an imaging optical system in which a peripheral region has large distortion with respect to a central region is a natural corrected image in which the user does not substantially feel distortion when viewing the image. In the apparatus, when correcting distortion of the image, an image capturing apparatus obtains a corrected image by correcting distortion at a fixed pixel interval for each pixel of the image.
(Supplementary Note 4) An image forming optical system in which a peripheral region has a large distortion with respect to a central region, and an image forming optical system in which a plurality of light receiving elements are two-dimensionally arranged is formed by the image forming optical system. An image sensor that photoelectrically converts an optical image with the plurality of light receiving elements and outputs an image signal; an image generating unit that generates image data of each pixel in the image of the optical image based on the image signal; An image for calculating image data of each pixel in the corrected image by correcting the image data of each pixel in the image of the light image so that the image of FIG. An image processing unit comprising: a processing unit configured to calculate image data of each pixel in the corrected image by performing distortion correction on image data of each pixel in the image of the light image at a fixed pixel interval. To An imaging device characterized by the following.
(Supplementary Note 5) The image processing unit includes a processing unit that reduces the image of the light image, a processing unit that corrects distortion of the reduced image, and a processing unit that enlarges the distortion-corrected image to the original image size. 5. The imaging device according to claim 4, wherein the imaging device includes: (Supplementary Note 6) The imaging apparatus according to Supplementary Note 3, wherein distortion is corrected for all pixels in a certain range of the image.
(Supplementary Note 7) The imaging apparatus according to Supplementary Note 4, wherein distortion correction is performed on all image data of each pixel within a certain range of the light image.
(Supplementary Note 8) The imaging device according to supplementary note 3, wherein an image of the light image transmitted through the central region is enlarged or reduced and used instead of an image within a certain range of the corrected image.
(Supplementary Note 9) The image data of each pixel within a certain range of the corrected image, wherein the image data of each pixel in the image of the light image transmitted through the central region is used without performing the distortion correction. An imaging device according to item 1.
(Supplementary Note 10) Distortion for correcting distortion so that an image obtained by an imaging optical system in which a peripheral region has a large distortion with respect to a central region becomes a natural corrected image in which the distortion is not substantially felt when viewed. A distortion correction method, comprising: a step of enlarging or reducing an image formed by an optical image transmitted through the central area; and a step of correcting distortion of an image formed by an optical image transmitted through the peripheral area.
(Supplementary Note 11) Distortion that corrects distortion so that an image obtained by an imaging optical system in which a peripheral region has a large distortion with respect to a central region becomes a natural corrected image in which the user does not substantially feel distortion when viewing the image. The correction method includes the steps of obtaining a corrected image by correcting distortion at a fixed pixel interval for each pixel of the image, and enlarging or reducing an image formed by a light image transmitted through the central region. Characteristic distortion correction method.
(Supplementary Note 12) Distortion for correcting distortion so that an image obtained by an imaging optical system in which a peripheral region has a large distortion with respect to a central region becomes a natural corrected image which does not substantially feel distortion when viewed. In the correction method, a step of correcting a distortion at a fixed pixel interval for each pixel of the image to obtain a corrected image, and a step of correcting the distortion for all pixels of each pixel within a predetermined range of the image. Performing the distortion correction.
[0107]
【The invention's effect】
As described above, since the imaging apparatus according to the present invention uses the optical system having a large distortion in the peripheral region with respect to the central region in the imaging optical system, not only can a wide-field image be obtained, The image of the light image transmitted through the central region becomes a natural image that does not substantially feel distortion when viewed. For this reason, since the image of the optical image transmitted through the central region is enlarged or reduced, and the enlarged or reduced image is used as a part of the corrected image, the distortion correction processing for correcting the distortion of the portion is not required. Thus, the distortion correction processing can be executed at high speed.
[0108]
Since the distortion correction processing can be executed at high speed, an inexpensive integrated circuit such as an LSI for general use or an ASIC for digital still camera can be used, so that the cost of the imaging apparatus can be reduced.
[0109]
In addition, since the distortion correction processing can be executed at a high speed, the frame rate can be improved as compared with the conventional case when the distortion correction processing is performed by a circuit having the same processing capability, so that a moving image having a smoother motion can be obtained. Can be obtained.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a configuration of a monitoring system according to a first embodiment.
FIG. 2 is a diagram illustrating a configuration of a monitoring camera used in the monitoring system.
FIG. 3 is a block diagram illustrating a configuration of a monitoring camera.
FIG. 4 is a diagram illustrating characteristics of the imaging optical system according to the first embodiment.
FIG. 5 is a block diagram illustrating a configuration of a controller.
FIG. 6 is a diagram illustrating an operation of the surveillance camera according to the first embodiment.
FIG. 7 is a diagram showing a display screen displayed on a display unit of the controller.
FIG. 8 is a diagram illustrating a distortion correction process.
FIG. 9 is a diagram illustrating an image after distortion correction processing in a telephoto case according to the first embodiment.
FIG. 10 is a diagram illustrating an image after distortion correction processing in the case of a wide angle in the first embodiment.
FIG. 11 is a diagram illustrating an operation of the surveillance camera when a wide-angle image is generated according to the second embodiment.
FIG. 12 is a diagram illustrating an image after distortion correction processing in a wide angle case according to the second embodiment.
FIG. 13 is a diagram illustrating an operation of the surveillance camera when a wide-angle image is generated according to the third embodiment.
FIG. 14 is a diagram illustrating a captured image captured by a wide-distortion lens and a distortion-corrected image obtained by correcting the distortion.
[Explanation of symbols]
2 surveillance cameras
3 Controller
30 Imaging unit
31 Image generator
32 image data buffer
33 Image processing unit
35, 43 control unit
36 Image data recording unit
37 Memory
41 Operation unit
42 Display

Claims (5)

In an imaging device that corrects distortion so as to become a natural corrected image that does not substantially feel distortion when viewing an image obtained by an imaging optical system whose peripheral region has a large distortion with respect to the central region,
An image pickup apparatus characterized in that an image of a light image transmitted through the central region is enlarged or reduced and used instead of correcting distortion in a part of the corrected image. In an imaging device that corrects distortion so as to become a natural corrected image that does not substantially feel distortion when viewing an image obtained by an imaging optical system whose peripheral region has a large distortion with respect to the central region,
An image pickup apparatus, wherein when correcting distortion of the image, a corrected image is obtained by correcting distortion at a fixed pixel interval for each pixel of the image. 3. The imaging apparatus according to claim 2, wherein for each pixel within a certain range of the image, distortion is corrected for all pixels. The imaging apparatus according to claim 2, wherein an image of a light image transmitted through the central region is enlarged or reduced and used instead of an image within a certain range of the corrected image. In a distortion correction method for correcting a distortion so as to become a natural corrected image in which the image obtained by the imaging optical system having a large distortion in the peripheral region with respect to the central region does not substantially feel distortion when viewed.
Step of enlarging or reducing the image by the light image transmitted through the central area,
Correcting the image of the light image transmitted through the peripheral region.

Images