JP2013080998A - Image processing device, image processing method, and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device, an image processing method, and an image processing program and the like that enable improvement in grasping an imaging target imaged as a distorted circle image.SOLUTION: A fisheye image is converted into fisheye image data that can be displayed on a fisheye image monitor 2. The fisheye image in a predetermined range based on a predetermined position is distortion-corrected into plane regular image data to be displayed on a plane regular image monitor 3 visibly in two dimensions. The fisheye image data or the plane regular image data is selectively outputted.

Description

  The present invention relates to a technical field such as an image processing apparatus and an image processing method for processing image data.

  A wide-angle lens (for example, a fisheye lens) or an omnidirectional mirror can capture a subject with a wide range of angles of view, and is therefore applied to a camera device (for example, a fisheye camera) such as a surveillance camera system or a video conference system.

  A fish-eye image (an example of a distorted circular image) captured by the fish-eye camera is distorted, and particularly, a significant distortion is generated at the outer edge (end face portion) of the image. Therefore, in order to improve the identifiability of the subject shown in the fisheye image, fisheye correction processing is performed in which the fisheye image is converted into a planar regular image that can be viewed two-dimensionally and the distortion is corrected.

  Also, as a technique to which fisheye correction processing is applied, in Patent Document 1, a fisheye lens whole view reduced image and a cutout distortion correction region of a desired region in the fisheye image designated by a pointing device are output and displayed on the same screen. Techniques to do this are disclosed.

JP 2008-301034 A

  However, in the technique disclosed in Patent Document 1, the fisheye lens whole view reduced image is reduced and superimposed on the image shown in the cutout distortion correction area, so that the fisheye lens whole view reduced image is sufficiently grasped. It was difficult. Moreover, since the fisheye lens whole view reduced image is reduced and superimposed on the image shown in the cutout distortion correction area, there is a problem that the degree of freedom in the display method of the image is low.

  Therefore, the present invention has been made in view of such problems and the like, and is an image processing apparatus, an image processing method, an image processing program, and the like that can improve the graspability of an imaging target imaged in a distorted circular image. Is one of the issues.

  In order to solve the above-described problem, the invention described in claim 1 converts a distorted circular image captured by a wide-angle lens or an omnidirectional mirror input from the outside into distorted circular image data that can be displayed on a display unit. A distorted circular image converting means; and a distortion correcting means for correcting the distortion of the distorted circular image within a predetermined range with reference to a predetermined position into planar regular image data that is two-dimensionally visible on the display unit; Selecting means for selectively outputting the distorted circular image data or the planar regular image data.

  According to the present invention, it is possible to improve the graspability of the imaging target imaged in the distorted circular image and the degree of freedom of the image display method.

  A second aspect of the present invention is the image processing apparatus according to the first aspect, wherein the selection unit displays the distorted circular image data on a first display unit and the planar regular image data on a second display. It outputs to each part, It is characterized by the above-mentioned.

  According to the present invention, it is possible to further improve the graspability of the imaging target imaged in the distorted circular image and the degree of freedom of the image display method.

  A third aspect of the present invention is the image processing apparatus according to the first or second aspect, further comprising instruction means for indicating a predetermined position of the distorted circular image, wherein the distortion correction means is the instruction means. The distorted circular image within a predetermined range with reference to the position designated by the above is distorted and output to planar regular image data displayed in a two-dimensional manner on the display unit so as to be visible.

  According to the present invention, it is possible to improve the graspability of an imaging target imaged in a distorted circular image and obtain a desired planar regular image from the distorted circular image.

  A fourth aspect of the present invention is the image processing apparatus according to any one of the first to third aspects, wherein the distorted circular image conversion means converts the distorted circular image into a predetermined magnification or size. It is characterized by converting into distorted circular image data.

  According to the present invention, it is possible to improve the graspability of an imaging target imaged in a circular image and obtain a desired planar regular image from a distorted circular image.

  A fifth aspect of the present invention is the image processing apparatus according to any one of the first to fourth aspects, wherein the distorted circular image data is converted as planar regular image data indicating a rectangular area. A pointer for displaying an area is displayed, and the instruction means can designate a predetermined position of the distorted circular image data by the pointer, and the distortion correction means can determine a predetermined position based on the position indicated by the pointer. The distorted circular image within the range is subjected to distortion correction to planar regular image data displayed in a two-dimensionally visible manner on the display unit, and is output.

  The invention according to claim 6 is an image processing method in the image processing apparatus, wherein distorted circular image data that can display a distorted circular image captured by a wide-angle lens or an omnidirectional mirror input from the outside on a display unit. A distorted circular image converting step for converting to a distorted circular image within a predetermined range with a predetermined position as a reference, and distortion correction for correcting the distortion into planar regular image data displayed in a two-dimensionally visible manner on the display unit And a selection step of selectively outputting the distorted circular image data or the planar regular image data.

  According to the seventh aspect of the present invention, a computer included in the image processing apparatus converts a distorted circular image captured by a wide-angle lens or an omnidirectional mirror input from the outside into distorted circular image data that can be displayed on the display unit. A distorted circular image converting means, a distortion correcting means for correcting the distortion of the distorted circular image within a predetermined range with reference to a predetermined position into planar regular image data displayed in a two-dimensionally visible manner on the display unit, The distorted circular image data or the planar regular image data is made to function as a selection unit that selectively outputs the distorted circular image data or the planar regular image data.

  According to the present invention, since the distorted circular image data or the planar regular image data is selectively output, it is possible to improve the graspability of the imaging target imaged in the distorted circular image.

1 is a system diagram illustrating an outline of a configuration and functions of an image processing system according to an embodiment. It is a block diagram which shows the structure and function outline | summary of the image processing apparatus 1 which concern on this embodiment. It is a conceptual diagram which shows conversion to the fish-eye image data in the image processing apparatus 1 to plane regular data. 3 is a conceptual diagram showing an outline of functions of a selector 17. FIG. It is an example of a display screen when it looks as if the fisheye image is displayed on the fisheye image monitor 2 and the planar regular image is displayed on the planar regular image monitor 3 at the same time. 3 is a flowchart showing a flow of processing executed by the image processing apparatus 1. When the fisheye image data is displayed at a predetermined magnification or size, and the fisheye image data within a predetermined range centered on a predetermined position of the fisheye image is converted into planar regular image data and displayed. It is a flowchart which shows operation | movement of the microcomputer 18 in the case of being. When the fisheye image data is displayed at a predetermined magnification or size, and the fisheye image data within a predetermined range centered on a predetermined position of the fisheye image is converted into planar regular image data and displayed. It is an example of a display screen in the case of. It is an example of a display screen which shows an example of another pointer. It is an example of a display screen when the area | region converted as planar regular image data which shows a rectangular area | region is previously displayed as a pointer on a fish-eye image. It is a block diagram which shows the structure at the time of adding the terminal device 4 to the image processing system S, and an outline of a function. It is an example of a display screen when one or a plurality of fisheye images or planar regular images are output on a divided screen and displayed on the display unit of the fisheye image monitor 2 or the planar regular image monitor 3.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The embodiment described below is an embodiment when the present invention is applied to an image processing system.

[1. Overview of image processing system configuration and functions]
First, with reference to FIG. 1, the configuration and functional overview of the image processing system according to the present embodiment will be described.

  FIG. 1 is a system diagram showing an outline of the configuration and functions of an image processing system according to this embodiment.

  As shown in FIG. 1, the image processing system S includes an image processing device 1 (an example of the image processing device of the present application), a fish-eye image monitor 2 (an example of the first display unit of the present application), and a planar regular image. Monitor 3 (an example of the second display unit of the present application) and the like. The image processing apparatus 1, the fish-eye image monitor 2, and the planar regular image monitor 3 may be wireless (for example, Bluetooth (registered trademark) or infrared communication) or wired (for example, electrical communication through a communication path such as an electric wire or an optical fiber). ) And the like (in detail, fisheye image data or planar regular image data described later) can be transmitted and received between each other.

  As will be described in detail later, the image processing apparatus 1 is a distorted circle that can display a fisheye image as an example of a distorted circular image captured by the optical system 10 and converted into a predetermined electrical signal on the fisheye image monitor 2. The image data is converted into fisheye image data as an example and output. Planar regular image data for displaying a fisheye image within a predetermined range with respect to a predetermined position (that is, part or all of the fisheye image) on the planar regular image monitor 3 so as to be two-dimensionally visible. This is a device that outputs a corrected distortion. Hereinafter, in the case of a simple fisheye image, the fisheye image data displayed on the fisheye image monitor 2 is used, and in the case of a flat regular image, the plane regular image data displayed on the flat regular image monitor 3 is displayed. It shall be said.

  Although the fish-eye image monitor 2 and the planar regular image monitor 3 are not shown in detail, for example, light emitted from a light source such as a backlight is partially blocked or transmitted by a polarizing filter, a liquid crystal, etc. Display for displaying images such as fisheye images and planar regular images with arbitrarily set brightness, display processing unit having CPU, RAM, ROM, etc., various data and fisheye image monitor 2 and planar regular images Reception (transmission / reception) of fisheye image data or planar regular image data between the image processing apparatus 1 and a storage unit having a hard disk (or stored in the ROM) in which a computer program responsible for the operation of the monitor 3 is stored. A communication unit for performing the above. The communication unit includes, for example, a DVI-D connector compliant with the DVI (Digital Visual Interface) standard, a USB connector compliant with the USB standard, a D-Sub connector, or the like, and the image processing apparatus 1 via these connectors. And is used to receive fish-eye image data or planar regular image data.

  As such a fish-eye image monitor 2 and planar regular image monitor 3, for example, a PDA (Personal Digital Assistant) can be applied.

  Further, as the fish-eye image monitor 2 and the planar regular image monitor 3, a display for a personal computer, a television monitor for general households, or the like can be applied. In this case, the fish-eye image monitor 2 and the planar regular image monitor 3 include only a display and a communication unit, and include a display processing unit and the like in an external device (the fish-eye image monitor 2 and the planar regular image monitor 3 Is not prepared).

  Note that in the drawings shown below, members having the same number have at least the same function and effect, and the description thereof is omitted.

[1-1. Overview of Configuration and Function of Image Processing Apparatus 1]
Next, the configuration and functional overview of the image processing apparatus 1 according to the present embodiment will be described with reference to FIG.

  FIG. 2 is a block diagram showing an outline of the configuration and functions of the image processing apparatus 1 according to this embodiment.

  As shown in FIG. 2, the image processing apparatus 1 according to the present embodiment includes an optical system 10, an enlargement / reduction circuit 11 (an example of a distorted circular image conversion unit of the present application), a memory controller 12, a memory 13, A correction circuit 14 (an example of the distortion correction unit of the present application), a memory controller 15, a memory 16, a selector 17 (an example of the selection unit of the present application), a microcomputer 18 (an example of the instruction unit of the present application), and the like. Has been.

  The optical system 10 converts a captured fisheye image into a predetermined electrical signal. Specifically, in the optical system 10, light taken from a fish-eye lens 10a as an example of a wide-angle lens or an omnidirectional mirror is adjusted by an aperture mechanism (not shown) and formed as an optical image on the image sensor 10b. . Such an optical image is converted into an electrical signal by the image sensor 10b and output. Hereinafter, the output electric signal is simply referred to as an input image.

  The fish-eye lens 10a is a kind of optical lens and has a field angle close to 180 degrees. There are four types of fisheye lens 10a: orthographic projection, equidistant projection, stereoscopic projection, and equisolid angle projection. In the present embodiment, such a method can be arbitrarily selected and is not limited to this. Furthermore, other wide-angle lenses or omnidirectional mirrors may be applied. The image sensor 10b converts the imaged optical image into a predetermined electrical signal proportional to the brightness of the optical image by a photoelectric exchange function using, for example, photodiodes arranged in a matrix and outputs the electric signal. As an example of the image sensor 10b, for example, a charge coupled semiconductor (CCD) image sensor, a complementary metal-oxide semiconductor (CMOS) image sensor, or the like is applied.

  The enlargement / reduction circuit 11 converts the input image output by the optical system 10 into fisheye image data that can be displayed on the fisheye image monitor 2 under the control of the microcomputer 18. The enlargement / reduction circuit 11 enlarges or reduces the captured fisheye image at a predetermined magnification (a magnification input by a not-shown operation unit or set in advance) or converts it into fisheye image data at a predetermined size (cutout size). Can be converted. The method for converting to fisheye image data is well known and will not be described in detail. For example, a predetermined video signal that can display the electrical signal on the fisheye image monitor 2, for example, NTSC system, PAL system, etc. Or converted into a progressive scan method or the like, or converted into a file (for example, PICT, JPEG, GIF, or MPEG) in a format that can be displayed on the fisheye image monitor 2 and output to the selector 17.

  The memory controller 12 transfers the picked-up fisheye image to the memory 13 constituted by, for example, a general buffer memory for storing image data, for example, at a position indicated by coordinates (x, y) on the XY coordinate system. It is stored as a pixel information group constituted by an aggregate of a large number of arranged pixels. Further, the memory controller 12 stores pixel data (for example, data indicating the respective luminances of R (red), G (green), and B (green)) at corresponding coordinates calculated by the correction circuit 14 described later. 13 is acquired from the frame image stored in 13 and output to the correction circuit 14.

  Under the control of the microcomputer 18, the correction circuit 14 converts all or part of the fisheye image into a planar regular image and displays a predetermined video signal that can be displayed on the planar regular image monitor 3, for example, the NTSC system, It converts into a PAL system, a progressive scan system, etc., outputs it, or converts into a file of a format which can be displayed on the fisheye image monitor 2.

  The conversion to the video signal may be performed after reading out the planar regular image data stored in the memory 16 to be described later under the control of the microcomputer 18.

  The conversion of all or part of the fisheye image into a planar regular image is a well-known technique, and thus detailed description is omitted. However, in order to obtain a pixel information group constituting the planar regular image, the planar regular image is configured. Corresponding coordinates where each pixel information is located in the fisheye image corresponding to each pixel information to be calculated are calculated.

  Specifically, a plane that touches an arbitrary point on a virtual sphere that models the optical characteristics of a fisheye lens is defined as a plane regular image, and each coordinate on the plane is converted into a known image by a known coordinate transformation. The coordinates are converted to the corresponding coordinates.

  Here, the corresponding coordinates are calculated by assuming that the point on the fisheye image is (x, y) and the point on the planar regular image is (u, v), and the correction coefficients A to F as parameters and the correction time are as follows. It is generally known that the coefficient w relating to the magnification is calculated by the equations (1) and (2).

また、平面正則画像Ｔ上の左右輪郭近傍に生じる歪みを改善するため、変換後の画素情報における座標Ｃ（ｕ、ｖ）に位置する画素情報を、ＵＶ湾曲座標系上の座標Ｃ´（ｕ´、ｖ）とＸＹ座標系上の座標Ｐ（ｘ、ｙ）との対応関係を示す対応関係式を用いて求めた座標Ｐ（ｘ、ｙ）の近傍に位置する画素情報に基づいて決定し、変換後の画素配列データに基づいて平面上の表示部に画像を表示することにより、平面正則画像を求める技術（詳しくは、例えば特開２０１０−６２７９０号公報参照）を適用して、魚眼画像から平面正則画像へと変換するようにしても良い。

In addition, in order to improve the distortion generated in the vicinity of the left and right contours on the planar regular image T, the pixel information located at the coordinates C (u, v) in the converted pixel information is converted into coordinates C ′ (u ', V) is determined based on pixel information located in the vicinity of the coordinates P (x, y) obtained by using a correspondence expression indicating the correspondence between the coordinates P (x, y) on the XY coordinate system. By applying a technique for obtaining a planar regular image by displaying an image on a planar display unit based on the converted pixel array data (for example, refer to Japanese Unexamined Patent Application Publication No. 2010-62790), the fisheye The image may be converted into a planar regular image.

  In this case, the corresponding coordinates are calculated by assuming that the point on the fisheye image is (x, y) and the coordinate on the UV curved coordinate system is C ′ (u ′, v), and correction coefficients A to F as parameters. And the coefficient w ′ relating to the magnification at the time of correction is calculated by the equations (3) and (4).

また、補正回路１４は、魚眼画像の一部を平面正則画像に変換する場合には、例えば、魚眼画像の切り出し位置、切り出し画像サイズ、倍率に応じた魚眼補正演算に必要な各パラメータの算出及び演算を行わせる。

In addition, when the correction circuit 14 converts a part of the fisheye image into a planar regular image, for example, each parameter necessary for fisheye correction calculation according to the cutout position, cutout image size, and magnification of the fisheye image. The calculation and calculation are performed.

  In this case, for example, with respect to the fisheye image displayed on the fisheye image monitor 2, the above-described parameters are calculated based on a predetermined position of the fisheye image instructed by an operation of an operation unit (not shown). The Then, the fisheye image data within a predetermined range is converted into planar regular data around a predetermined position of the fisheye image.

  Here, with reference to FIG. 3, conversion of fisheye image data in the image processing apparatus 1 to plane regular data will be described.

  FIG. 3 is a conceptual diagram illustrating conversion of fisheye image data into planar regular data in the image processing apparatus 1.

  For example, as shown in FIG. 3A, a fisheye image 22 is displayed on the display screen 21 of the fisheye image monitor 2. The fisheye image 22 displays a pointer 23 that can select a predetermined position of the fisheye image by a user operation or the like.

  When a predetermined position of the fisheye image is selected by the pointer 23, the correction circuit 14 converts the fisheye image data within a predetermined range around the pointer 23 into planar regular image data.

  Then, as shown in FIG. 2B, the converted planar regular image data is displayed on the display unit 31 of the planar regular image monitor 3.

  Note that the correction circuit 14 may convert all of the input fisheye image data into planar regular image data regardless of the user's selection operation.

  Returning to the description of FIG. 2, the memory controller 15 transfers the planar regular image data converted by the correction circuit 14 to, for example, a memory 16 constituted by a general buffer memory for storing image data on the UV coordinate system. It is stored as a pixel information group constituted by an aggregate of a large number of pixels arranged at the position indicated by coordinates (u, v). The contour of the planar regular image data can be arbitrarily set.

  The selector 17 selectively outputs predetermined data to a predetermined device based on the control of the microcomputer 18. In the present embodiment, the selector 17 outputs fisheye image data to the fisheye image monitor 2 and planar regular image data to the planar regular image monitor 3 based on the control of the microcomputer 18.

  Here, the functional outline of the selector 17 will be described with reference to FIG.

  FIG. 4 is a conceptual diagram showing a functional outline of the selector 17.

  As shown in FIG. 4, the selector 17 includes a lever L that can be rotated around P based on the control of the microcomputer 18, a terminal A and a terminal A ′ that are electrically connected by the rotation of the lever L, and Terminal B and terminal B ′ are provided.

  The terminal A is an end of a signal line that is electrically connected to the enlargement / reduction circuit 11 and outputs fisheye image data. The terminal A ′ is an end of a signal line that is electrically connected to the fish-eye image monitor 2 via the microcomputer 18. When the terminal A and the terminal A ′ are connected by the lever L, fisheye image data is output to the fisheye image monitor 2 and a fisheye image is displayed.

  The terminal B is an end of a signal line that is electrically connected to the memory controller 15 and outputs planar regular image data. The terminal B ′ is an end portion of a signal line that is electrically connected to the planar regular image monitor 3 via the microcomputer 18. When the terminal B and the terminal B ′ are connected by the lever L, the planar regular image data is output to the planar regular image monitor 3, and the planar regular image is displayed.

  In the image processing apparatus 1, the fisheye image can be displayed on the fisheye image monitor 2 and the planar regular image can be displayed on the planar regular image monitor 3 at a predetermined timing based on the control of the microcomputer 18.

  Further, by switching the lever L at high speed based on the control of the microcomputer 18, the state where the terminal A and the terminal A ′ are connected and the state where the terminal B and the terminal B ′ are connected can be alternately switched at high speed. It is possible to make it appear as if the fisheye image is displayed on the fisheye image monitor 2 and the planar regular image is displayed on the planar regular image monitor 3 as if they are simultaneously displayed.

  Here, referring to FIG. 5, an example of a display screen when a fisheye image is displayed on the fisheye image monitor 2 and a planar regular image is displayed on the planar regular image monitor 3 as if they are simultaneously displayed. Will be described.

  FIG. 5 is an example of a display screen when the fish-eye image appears to be displayed on the fish-eye image monitor 2 and the planar regular image appears on the planar regular image monitor 3 as if they are simultaneously displayed.

  As shown in FIG. 5, under the control of the microcomputer 18, the state where the terminal A and the terminal A ′ are connected and the state where the terminal B and the terminal B ′ are connected based on the high-speed switching of the lever L. By alternately switching at high speed, it looks as if the fisheye image is displayed on the fisheye image monitor 2 and the planar regular image is displayed on the planar regular image monitor 3 at the same time.

  Such a selector 17 may be performed by a hardware circuit (device) mounted in the image processing apparatus 1, or a microcomputer (such as the microcomputer 18) mounted in the image processing apparatus 1. It may be performed by executing a predetermined program (software) stored in a ROM or the like.

  Returning to FIG. 2, the microcomputer 18 includes, for example, a CPU, a ROM (for example, a control program or the image processing program of the present application is stored), a RAM, and the like, and comprehensively controls the entire image control apparatus 1. The enlargement / reduction circuit 11, the correction circuit 14, and the selector 17 are configured to perform the above-described operation.

  The image processing apparatus 1 according to the present invention is an apparatus having a function of converting a part of a fisheye image obtained by photographing using a fisheye lens into a planar regular image. An image to be converted by this apparatus. Is not limited to images obtained by photographing using a fisheye lens. For example, if an image to which a hemispherical projection model equivalent to a fisheye lens is applied, such as a photographed image using a convex mirror (such as a wide-angle lens or an omnidirectional mirror), image conversion using the image processing apparatus according to the present invention is possible. It is.

  Further, such an operation (processing) of the image processing apparatus 1 (the microcomputer 18 or the like) may be performed by a hardware circuit mounted in the image processing apparatus 1, or may be performed in the image processing apparatus 1. The microcomputer (such as the microcomputer 18) to be mounted may be executed by executing a predetermined program (software) stored in a ROM or the like.

[2. Operation of image processing apparatus 1]
Next, the operation of the image processing apparatus 1 will be described with reference to FIGS.

  First, the flow of processing executed by the image processing apparatus 1 will be described with reference to FIG.

  FIG. 6 is a flowchart showing a flow of processing executed by the image processing apparatus 1.

  As shown in FIG. 6, when an input image (fisheye image) captured by the fisheye lens 10a and output by the optical system 10 is input (step S1), the input image is displayed on the enlargement / reduction circuit 11 and the memory controller. 12 is input.

  When this input image is input to the enlargement / reduction circuit 11 (step S2), the enlargement / reduction circuit 11 enlarges or reduces the captured fisheye image at a predetermined magnification, or fisheye image data of a predetermined size. (Step S3) and output to the selector 17.

  When the input image is input to the memory controller 12, the memory controller 12 stores the input image in the memory 13 (step S4).

  Then, for example, the memory controller 12 acquires, as necessary, pixel data at the corresponding coordinates calculated by the correction circuit 14 from the frame image stored in the memory 13, and outputs the data to the correction circuit 14. 14 converts into plane regular image data (step S5).

  Then, the memory controller 15 stores the planar regular image data converted by the correction circuit 14 in the memory 16 (step S6).

  Based on the control of the microcomputer 18, the selected fisheye image data or planar regular image data is output from the selector 17 (step S7).

  Then, the fisheye image data is displayed on the fisheye image monitor 2, and the planar regular image data is displayed on the planar regular image monitor 3 (step S8).

  Next, referring to FIG. 7 and FIG. 8, when the fisheye image data is displayed at a predetermined magnification (including enlargement or reduction) or size, and around a predetermined position of the fisheye image, The operation of the microcomputer 18 when fisheye image data within a predetermined range is converted into planar regular image data and displayed will be described.

  FIG. 7 shows a case where fisheye image data is displayed at a predetermined magnification or size, and fisheye image data within a predetermined range is converted into planar regular image data around a predetermined position of the fisheye image. It is a flowchart which shows operation | movement of the microcomputer 18 in the case of being displayed.

  FIG. 8 shows a case where fisheye image data is displayed at a predetermined magnification or size, and fisheye image data within a predetermined range is converted into planar regular image data around a predetermined position of the fisheye image. It is an example of a display screen in the case of being displayed.

  As shown in FIG. 7, for example, when a fisheye image is displayed at a predetermined magnification or size, the user performs a predetermined magnification (for example, a predetermined enlargement ratio or Enter a reduction ratio or the like) or size (image size or the like). Further, when fisheye image data within a predetermined range is converted into planar regular image data and displayed, the user can specify a numerical value (for example, a predetermined range) based on an operation of an operation unit (not shown). The image of the planar regular image after conversion is input.

  The input value is transmitted as a parameter to the enlargement / reduction circuit 11 or the correction circuit 14 (step S11).

  Specifically, when a fisheye image is displayed at a predetermined magnification or size, the enlargement / reduction circuit 11 uses the input predetermined magnification or size from the fisheye image data based on the input parameters. Fish-eye image data corresponding to the size is extracted.

  The display screen example shown in FIG. 8A is a display screen example when a fisheye image is displayed at a predetermined magnification or size from fisheye image data.

  In FIG. 8A, an image within a predetermined range centered on the person who has been imaged is displayed in the fisheye image data.

  When fisheye image data within a predetermined range is displayed after being converted into planar regular image data, the correction circuit 14 determines the cutout position and cutout image size of the fisheye image based on the input parameters. Then, each parameter necessary for fisheye correction calculation according to the magnification is calculated and calculated, and the corresponding coordinates described above are calculated.

  The display screen example shown in FIG. 8B is a display screen example when fisheye image data within a predetermined range is converted into planar regular image data and displayed.

  In FIG. 8B, an image within a predetermined range centered on the building is displayed.

  Then, the microcomputer 18 selects and acquires an image (image data) from the selector 17 (step S12).

  Then, the image is output to the outside (step S13).

  For example, based on the control of the microcomputer 18, the selector 17 outputs the fisheye image data to the fisheye image monitor 2 and the planar regular image data to the planar regular image monitor 3.

  When there is a parameter change (step S14: YES), the process proceeds to step S11. When there is no parameter change (step S14: NO), the process proceeds to step S12. .

  As described above, in the present embodiment, a fisheye image is converted into fisheye image data that can be displayed on the fisheye image monitor 2, and a fisheye image within a predetermined range with a predetermined position as a reference is converted. Since the fisheye image data or the planar regular image data is selectively output by correcting the distortion to the planar regular image data displayed in a two-dimensionally recognizable manner on the planar regular image monitor 3, the fisheye It is possible to improve the graspability of the imaging target imaged in the image and the degree of freedom of the image display method.

  Further, since the fisheye image data is output to the fisheye image monitor 2 and the planar regular image data is output to the planar regular image monitor 3 based on the control of the microcomputer 18, the imaging to be captured in the fisheye image is performed. The graspability of the object and the degree of freedom of the image display method can be improved.

  Further, the fisheye image data within a predetermined range with reference to the position indicated by the pointer 23 is subjected to distortion correction to the planar regular image data displayed in a two-dimensionally visible manner on the planar regular image monitor 3 and output. Since it comprised in this way, the graspability of the imaging target imaged to a fisheye image can be improved, and a desired plane regular image can be obtained from a fisheye image.

  Further, since the fisheye image is configured to be converted into fisheye image data at a predetermined magnification or size, it improves the graspability of the imaging target imaged in the fisheye image, and the desired planar regularity is obtained from the fisheye image. An image can be obtained.

  Further, the pointer described above can take various forms.

  Here, another example of the pointer will be described with reference to FIGS. 9 and 10.

  FIG. 9 is an example of a display screen showing an example of another pointer. FIG. 10 shows a display when an area to be converted as planar regular image data indicating a rectangular area is previously displayed as a pointer on a fisheye image. It is an example of a screen.

  As shown in FIG. 9A, the pointer may be displayed with a cross mark 35, or may be displayed with a predetermined rectangular area 36 as shown in FIG. 10B.

  Further, as shown in FIG. 10, an area converted as planar regular image data indicating a rectangular area is displayed by a pointer 37 (an example of the pointer of the present application) that is displayed as a pointer on a fisheye image in advance. Also good.

  As described above, in order to convert a distorted fish-eye image into a planar regular image, each pixel in the fish-eye image (u, v) corresponding to each pixel (u, v) in the planar regular image is converted based on the above parameters. x, y) are specified.

  Here, consider a case where a fisheye image is converted into a planar regular image showing a rectangular region.

  When each pixel in the fisheye image corresponding to each pixel constituting the planar regular image in this case is schematically represented on the fisheye image, as shown by a pointer 37 in FIG. 10, a distorted rectangle is obtained. .

  Therefore, the pointer 37 in FIG. 10 displays an area to be converted as planar regular image data indicating a rectangular area as a pointer on the fisheye image in advance.

  As a result, when obtaining a planar regular image showing a desired rectangular area from the imaging target shown in the fisheye image, it is possible to grasp the area to be converted as a planar regular image simply by looking at the fisheye image. Furthermore, it is possible to easily specify an imaging target and obtain a desired planar regular image.

  When the area indicated by the pointer 37 in FIG. 10 is converted as planar regular image data representing a rectangular area, for example, a planar regular image indicated by the display unit 31 in FIG. 3 is displayed.

  Further, the terminal device 4 may be added to the image processing system S.

  Here, a configuration in which the terminal device 4 is added to the image processing system S will be described with reference to FIGS. 11 and 12.

  First, with reference to FIG. 11, the configuration and functional overview when the terminal device 4 is added to the image processing system S will be described.

  FIG. 11 is a block diagram showing an outline of the configuration and functions when the terminal device 4 is added to the image processing system S.

  As shown in FIG. 11, the image processing apparatus 1 is connected to a fish-eye image monitor 2 and a planar regular image monitor 3 via a terminal device 4.

  The terminal device 4 includes an operation unit (not shown), a display unit, a communication unit, a storage unit, an input / output interface unit, and a system control unit (including a CPU, ROM, RAM, and the like). A computer, a PDA (Personal Digital Assistant), a mobile phone, or the like can be applied. Then, based on the control of a software program stored in a ROM or the like, for example, display control of divided screens shown below is performed.

  Then, the terminal device 4 outputs one or a plurality of fisheye images or planar regular images on the divided screens and displays them on the display unit of the fisheye image monitor 2 or the planar regular image monitor 3.

  Here, referring to FIG. 12, one or a plurality of fisheye images or planar regular images are output to the divided screens and displayed on the display unit of the fisheye image monitor 2 or planar regular image monitor 3. An example display screen will be described.

  FIG. 12 shows an example of a display screen when one or a plurality of fisheye images or planar regular images are output on the divided screens and displayed on the display unit of the fisheye image monitor 2 or the planar regular image monitor 3. is there.

  Specifically, as shown in FIG. 12, a screen divided into three parts is displayed on the display unit of the fish-eye image monitor 2, and specifically, the entire captured fish-eye image is displayed. The fish-eye image 41 shown, the planar portion 42 of the fish-eye image 41 that has been enlarged and converted into a planar regular image, and the tower portion of the fish-eye image 41 that has been enlarged and transformed into a planar regular image A planar regular image 43 is displayed.

  Further, as shown in FIG. 12, the display unit of the planar regular image monitor 3 displays a screen divided into four parts. Specifically, the building portion of the fish-eye image 41 is enlarged. A plane regular image 44 converted into a plane regular image, a plane regular image 45 in which a person of the fisheye image 41 is enlarged and converted into a plane regular image, and a car portion of the fisheye image 41 is enlarged to a plane regular image. The converted flat regular image 46 and the flat regular image 47 obtained by enlarging the steel tower portion of the fish-eye image 41 and converting it into a flat regular image are displayed.

  Moreover, as a display mode of the divided screen (divided screen), the layout, the size of the divided screen, etc. are freely changed for each divided screen, or a part or all of each divided screen is displayed in a superimposed manner. It is also possible to apply so-called window display.

  The embodiments described above do not limit the invention according to the claims. And not all the combinations of the configurations described in the above embodiment are indispensable means for solving the problems of the invention.

  In the above embodiment, an example in which the present application is applied to an image processing system has been shown. However, for example, for a digital video camera, a digital camera, a personal computer, a household electronic device, etc. Is applicable.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 2 Fish-eye image monitor 3 Planar regular image monitor 4 Terminal apparatus 10 Optical system 11 Enlargement / reduction circuit 12 Memory controller 13 Memory 14 Correction circuit 15 Memory controller 16 Memory 17 Selector 18 Microcomputer S Image processing apparatus system

Claims (7)

A distorted circular image conversion means for converting a distorted circular image captured by an externally input wide-angle lens or an omnidirectional mirror into distorted circular image data that can be displayed on the display unit;
Distortion correcting means for correcting distortion of the distorted circular image within a predetermined range with a predetermined position as a reference to planar regular image data displayed in a two-dimensionally visible manner on the display unit;
Selection means for selectively outputting the distorted circular image data or the planar regular image data;
An image processing apparatus comprising: The image processing apparatus according to claim 1,
The image processing apparatus, wherein the selection unit outputs the distorted circular image data to a first display unit and the planar regular image data to a second display unit. The image processing apparatus according to claim 1, wherein:
Further comprising instruction means for indicating a predetermined position of the distorted circular image data;
The distortion correction means corrects distortion of the distorted circular image within a predetermined range on the basis of the position instructed by the instruction means into planar regular image data that is displayed two-dimensionally on the display unit. An image processing apparatus for outputting. An image processing apparatus according to any one of claims 1 to 3,
The distorted circular image conversion means converts the distorted circular image into distorted circular image data at a predetermined magnification or size. The image processing apparatus according to any one of claims 1 to 4,
In the distorted circular image data, a pointer for displaying an area to be converted as planar regular image data indicating a rectangular area is displayed,
The instruction means can indicate a predetermined position of the distorted circular image by the pointer,
The distortion correction unit corrects distortion of the distorted circular image data within a predetermined range with reference to the position indicated by the pointer into planar regular image data that is displayed in a two-dimensional manner on the display unit. An image processing apparatus for outputting. An image processing method in an image processing apparatus,
A distorted circular image conversion step of converting a distorted circular image captured by a wide-angle lens or an omnidirectional mirror input from the outside into distorted circular image data that can be displayed on the display unit;
A distortion correction step of correcting distortion of the distorted circular image within a predetermined range with a predetermined position as a reference to planar regular image data displayed in a two-dimensionally visible manner on the display unit;
A selection step of selectively outputting the distorted circular image data or the planar regular image data;
An image processing method comprising: A computer included in the image processing apparatus;
A distorted circular image conversion means for converting a distorted circular image captured by a wide-angle lens or an omnidirectional mirror input from the outside into distorted circular image data that can be displayed on the display unit;
Distortion correcting means for correcting distortion of the distorted circular image within a predetermined range with a predetermined position as a reference to planar regular image data displayed in a two-dimensionally visible manner on the display unit;
Selection means for selectively outputting the distorted circular image data or the planar regular image data,
An image processing program that functions as an image processing program.

Images