JP2010102423A - Wide angle image display method and wide angle image display system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a wide angle image display system for precisely calculating the image center of a wide angle image. <P>SOLUTION: An imaging step images a chart for compensating by means of an imaging means having a wide angle lens to acquire image data. A chart for compensating area detection step detects an area of the chart for compensation on an image. A circular dot detection step obtains a center point in a circular dot area on the imaged chart for compensation. An association step makes pixels on the image associated with points on the chart for compensation. A straight line fitting step selects a plurality of straight lines on which errors reach values not larger than a threshold if the points fit a point group aligned on a straight line on the chart for compensation. An image center calculation step sets an intersection point of a selected plurality of straight lines to the image center of an all-round direction. An imaging step acquires image data by imaging by the imaging means. A distortion correction step corrects distortion on the basis of the image center calculated by compensation processing. A display step displays a distortion-corrected image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a wide-angle image display method and a wide-angle image display system, and more particularly to a wide-angle image display method and a wide-angle image display system using a calibration chart.

  Conventionally, a wide-angle image display system includes a wide-angle imaging device, an image processing device, and a display device, acquires an image from the wide-angle imaging device, calculates an image center in a circular wide-angle image by the image processing device, Distortion correction is performed based on the image center coordinates, and the image is displayed by a display device. As an image center calculation method, there is a method based on a marker printed on a wide-angle imaging device (Patent Document 1, Patent Document 2).

In addition, there is a method using a histogram of pixel values in a circular area (Patent Documents 3 and 4). In addition, there is a method based on contour information of a circular area (Patent Document 5).
Further, as a wide-angle imaging device, a device using a wide-angle lens having a reflecting surface formed by molding a transparent material such as glass or resin and partially depositing a metal such as aluminum is known (Patent Document 6). .
JP 2003-304532 A JP 2003-303335 A JP 2003-303348 A JP 2003-303347 A JP 2003-303342 A JP 2005-109785 A

However, the method of calculating the center position based on the marker printed on the wide-angle lens of the wide-angle imaging device requires a step of printing the marker on the curved surface of the lens with high accuracy in the manufacture of the wide-angle imaging device. There is a problem that becomes larger.
In addition, the method of detecting the image center position based on the histogram of pixel values and the method based on the contour information of the circular area have a problem that the calculated coordinates are easily affected by image noise. In addition, the region on the image does not have a perfect circle due to a defect caused in manufacturing, and the preconditions of these methods are often not satisfied.
The present invention has been made in view of these problems, and provides a wide-angle image display method capable of calculating the image center of a wide-angle image with high accuracy.

  According to the first aspect of the present invention, an imaging step of capturing a calibration chart by an imaging unit having a wide-angle lens to acquire image data, and a calibration chart area for detecting a calibration chart area on the image A detection step, a circular dot detection step for obtaining a center point of a circular dot region on the captured calibration chart, an association step for associating a pixel on the image with a point on the calibration chart, and a straight line on the calibration chart A line fitting step for selecting a plurality of straight lines whose error is equal to or less than a threshold when fitting to a group of points arranged in an image center, and an image center calculating step using the intersection of the selected plurality of straight lines as the image center of the omnidirectional image, A calibration process comprising: an imaging step of acquiring image data by imaging with the imaging means; and the calibration process. Those having a distortion correction step of correcting a distortion based on the image center calculated, and a display step of displaying an image distortion correction, a display processing consisting of the.

  Further, according to the present invention, as described in claim 2, in the imaging step, the calibration chart is a plane on which regularly arranged patterns are printed.

  According to the present invention, as described in claim 3, in the imaging step, the calibration chart is a flat display on which regularly arranged patterns are displayed.

  According to a fourth aspect of the present invention, there is provided an imaging step for capturing a calibration chart by an imaging means having a wide-angle lens to acquire image data, and a calibration for detecting a calibration chart area on the image. On the calibration chart, a chart area detection step, a circular dot detection step for obtaining a center point of a circular dot area on the captured calibration chart, an association step for associating a pixel on the image with a point on the calibration chart, A straight line fitting step for selecting a plurality of straight lines whose error is equal to or less than a threshold when fitting to a group of points arranged on a straight line, and an image center calculating step using the intersection of the selected plurality of straight lines as the image center of the omnidirectional image A calibration mode, an imaging step of acquiring image data by imaging with the imaging means, A distortion correction step of correcting a distortion based on the image center calculated in the calibration mode, a display step of displaying an image distortion correction, and a display mode having a, but with the.

  According to the present invention, as described in claim 5, in the imaging step, the calibration chart is a plane on which regularly arranged patterns are printed.

  Further, according to the present invention, as described in claim 6, in the imaging step, the calibration chart is a flat display on which regularly arranged patterns are displayed.

  The image center can be calculated with high accuracy, and an image with small distortion can be displayed.

  Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. The wide-angle image display system includes a wide-angle imaging device 1, an image processing device 2, a display device 3, and a control device 4 (see FIG. 1). As shown in FIG. 2, the wide-angle image display system determines the operation mode (step S1), performs display processing (step S2) or calibration processing (step S3) corresponding to the display mode or calibration mode, It operates to repeat these.

  As shown in FIG. 3, in the display process S2, image data captured by the wide-angle imaging device 1 is acquired (step S4), distortion correction is performed on the acquired image by the image processing device 2 (steps S5 and S6), and The distortion corrected image is displayed by the display device 3 (step S7).

  The wide-angle imaging device 1 includes a wide-angle lens 1a having a curved mirror and an imaging element 1b made of a CCD, a CMOS, or the like (see FIG. 4). The wide-angle imaging device 1 projects light incident on the wide-angle lens 1a onto the imaging surface of the imaging device 1b by reflection by the curved mirror and refraction by the entrance surface and the exit surface, and the imaging device 1b outputs the captured image data. .

In the wide-angle imaging device 1 using the wide-angle lens 1a having a curved mirror, the captured image 5 is generally circular as indicated by reference numeral 5a (see FIG. 5). Reference numeral 5b denotes an image when a square surface is captured by the wide-angle imaging device 1, and the square image is an image in which each side is deformed into a curve due to distortion by the wide-angle lens 1a.
The corrected square image 6a in the distortion corrected image 6 obtained by correcting the distortion of the captured image 5 is converted into a square without distortion as a result of image processing (see FIG. 6).

  FIG. 7 is a block diagram illustrating a configuration of the image processing apparatus 2. The image processing device 2 receives the wide-angle image output from the wide-angle imaging device 1, performs distortion correction on the wide-angle image, and converts the wide-angle image into an image in which the wide-angle distortion is corrected. The image processing apparatus 2 includes a so-called microcomputer, and includes a CPU 2a, a ROM 2b, a general-purpose memory 2c, a video memory 2d, and an external device I / F 2e.

  The CPU 2a reads and executes a predetermined program stored in advance in the ROM 2b, temporarily stores an input image, a processed image, a calculation result, and the like from the wide-angle imaging device 1 in the general-purpose memory 2c and causes the display device 3 to display them. The processed image is stored in the video memory 2d. The wide-angle imaging device 1, the display device 3, and the control device 4 are connected to each part of the image processing device 2 via the external device I / F 2e.

FIG. 8 is a flowchart of the distortion correction process. The image processing device 2 performs the following processing for all pixels in the input image. The input image is an arrangement of M rows and N columns of luminance value digital data with the upper left corner as the origin, and the distortion corrected image has the same format. (X ′, y ′) is calculated from coordinates (x, y) on the input image by coordinate transformation described later (step S10). Next, as shown in Expression 1, the pixel value is replaced (step S11).

Here, I (x, y) and I ′ (x, y) are pixel values of pixels at the coordinates (x, y) of the input image and the distortion-corrected image, respectively. It shall take the value of I _max. These are repeatedly executed for x = 0, 1, 2,... N-1, y = 0, 1, 2,.

In FIG. 9, the pixel 7a of the input image 7 is moved to the pixel 7b by coordinate conversion. Assuming that the coordinates of the target pixels 7a and 7b are (x, y) and (x ′, y ′), respectively, their relations are as shown in Expressions 2 to 5.

Here, r is a distance from the image center (x ₀ , y ₀ ) calculated by a calibration process described later to the target pixel, and r ′ is a distance after performing a conversion f described later on r. , Θ is an angle formed by a line drawn from the image center 7c to the target point and the x axis (counterclockwise is a positive direction).

  FIG. 10 is an explanatory diagram illustrating an example of the conversion f. In FIG. 10, the left side and the right side of the table are the value before conversion and the value after conversion, respectively. For example, r = 2 becomes r ′ = 1.98 after conversion.

  FIG. 11 is a schematic diagram of the calibration chart 8. The calibration chart 8 is obtained by printing a diagram in which circular dots 8a are arranged in a lattice pattern on a square flat plate. The central circular dot 8b and the reference vertex circular dot 8c are printed in different colors so that they can be distinguished from other circular dots.

FIG. 12 is a flowchart showing the calibration process S3, and FIGS. 13 and 14 are schematic diagrams for explaining the calibration process S3.
In the imaging step S14, an image obtained by capturing the calibration chart 8 by the wide-angle imaging device 1 is acquired. In the chart area detection step S15, a calibration chart area on the image is detected. In step S16, it calculates the area and the ratio of the circumferential length of the detected chart area, if the ratio is smaller than the threshold value T _R is finished processing, greater than the threshold T _R, the flow proceeds to circle dot detection step S17.

  In the circular dot detection step S17, the center coordinates of the circular dots on the image are calculated. In step S18, it is determined whether the number of points is a prescribed number. If the number is not the prescribed number, the process is terminated, and if it is the prescribed number, the process proceeds to a circular dot associating step S19. In the circular dot association step S19, the detected center coordinates are associated with each circular dot in the calibration chart 8.

  In the straight line fitting step S20, for a group of circular dots whose center coordinates are on the same straight line on the calibration chart 8, straight line fitting is performed on the center coordinates of the circular dots corresponding to each element in the group. As a fitting method, a known algorithm such as a least square method is applied, and two dot groups are selected in the order of the smallest fitting error. Finally, the intersection of the straight lines corresponding to each group is calculated by a known geometric solution (step S21). The calculated intersection is the center of the image to be obtained.

Next, the chart area detection step S15 will be described in detail with reference to FIG. In the chart area detection step S15, first, the input image is binarized by the luminance value (step S22). Binarization can take place by threshold processing is a known technique, given threshold, for example, _{I max} / 2 following pixel values and _{I max} / 2 greater than the pixel value, respectively, 0 and _{I max} Assign pixel values. Hereinafter, regions having pixel values of 0 or I _max are referred to as a dark region and a bright region, respectively. Next, the area (number of pixels) of the bright region in the binarized image is calculated, and a region where it is larger than a predetermined threshold T _S , for example, M × N / 4 is detected (step S23). If the image is captured so that the chart area on the image is ¼ or more of the screen, the detected area is the chart area.

  Next, the circular dot detection step S17 will be described in detail with reference to FIG. In the circular dot detection step S17, first, individual dot areas are identified from the dot areas in the chart area by a labeling process which is a well-known technique. Further, as shown in FIG. 11, since the chart center 8b and the reference vertex 8c are different in color from other circular dots, they are also detected by color at the same time (step S24). Here, the number of all detected circular dots is K. For each of the K circular dot regions, the barycentric coordinates in the binary image, which is a well-known technique, are calculated (step S25). The calculated center of gravity is set as the detected center coordinate.

Next, based on FIG.17 and FIG.18, circular dot matching step S19 is explained in full detail.
In the circular dod association step S19, first, the uncorrelated point list is initialized to be all the center points detected in the circular dot detection step S17 (step S27). As shown in FIG. 18, the circular dots are arranged on a plurality of square sides with the chart center as a common center. These are hereinafter referred to as concentric squares. In the case of FIG. 18, the number of concentric squares is three. Further, the following processing is repeatedly executed for the number L of concentric squares.

  First, the outermost point sequence (points on the concentric square 8d in the case of FIG. 18) is associated by an outer concentric square association process described later (step S28). Next, the uncorrelated point list is updated by removing the point sequence associated by the outer concentric square association processing S28 from the uncorrelated point list (step S29). Through these repeated executions, all center points are associated with circular dots on the chart.

  FIG. 19 is a flowchart showing the processing contents of the outer concentric square association processing S28. In the outer concentric square association process S28, first, the concentric square list is initialized to be empty (step S31). Next, a binary image indicating the chart area is copied to the temporary work area, and the chart area image is subjected to contour extraction, which is a well-known binary image processing technique (step S32). In step S33, it is determined whether a dot area has been detected. If a pixel belonging to the circular dot region is detected on the contour line, the center of the circular dot is added to the concentric square list (steps S34 and S35). Next, the contour line is erased from the chart area (step S36). These steps S32 to S36 are repeated until all the outer dot areas are found (step S37). Finally, since the center coordinates found by the above processing are irregularly arranged, they are sorted in a fixed direction (for example, clockwise) with respect to the center of the chart area (step S38).

According to this embodiment, the image center can be calculated with high accuracy by the image processing device 2, and a wide-angle image with small distortion can be displayed on the display device 3.
Although the calibration chart of the present embodiment is a figure printed on a plane, the calibration chart may be a chart displayed on a liquid crystal display device, for example.

The block diagram which shows embodiment of this invention. The flowchart which shows operation | movement of embodiment same as the above. The flowchart which shows the display process of embodiment same as the above. Sectional drawing which shows the wide angle imaging device of embodiment same as the above. Image explanatory drawing of the wide-angle imaging device input image of embodiment same as the above. Image explanatory drawing of the distortion correction image of embodiment same as the above. The block diagram which shows the image processing apparatus of embodiment same as the above. The flowchart which shows operation | movement of the image processing apparatus of embodiment same as the above. Explanatory drawing of the distortion correction process of embodiment same as the above. Explanatory drawing of the conversion table of the distortion correction process of embodiment same as the above. The schematic diagram of the chart for calibration of embodiment same as the above. The flowchart of the calibration process of embodiment same as the above. The schematic diagram of the calibration process of embodiment same as the above. The schematic diagram of the calibration process of embodiment same as the above. The flowchart of the chart area | region detection step of embodiment same as the above. The flowchart of the center detection step of embodiment same as the above. The flowchart of the matching step of embodiment same as the above. Explanatory drawing of the matching step of embodiment same as the above. The flowchart of the outer concentric square matching process of embodiment same as the above.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Wide angle imaging device 1a Wide angle lens 1b Image pick-up element 2 Image processing device 2a CPU
2b ROM
2c General-purpose image memory 2d Video memory 2e External device I / F
DESCRIPTION OF SYMBOLS 3 Display apparatus 4 Operation means 5 Input image 5a Ring image 5b Square image before correction | amendment 6 Correction image 6a Square image after correction | amendment 7 Correction | amendment image coordinate system 7a Uncorrected coordinate 7b Corrected coordinate 7c Image center coordinate 8 Calibration chart (calibration chart) )
8a Circular dot 8b Chart center circular dot 8c Reference vertex circular dot

Claims (6)

An imaging step of capturing a calibration chart by an imaging means having a wide-angle lens and acquiring image data, a calibration chart area detecting step of detecting a calibration chart area on the image, and a circular dot on the captured calibration chart A circular dot detection step for obtaining the center point of the region, an associating step for associating the pixels on the image with the points on the calibration chart, and an error when the fitting is performed on a point group arranged on a straight line on the calibration chart. A calibration process comprising: a straight line fitting step for selecting a plurality of straight lines as described below; and an image center calculating step using the intersection of the selected plurality of straight lines as the image center of the omnidirectional image;
An imaging step of acquiring image data by imaging by the imaging means, a distortion correction step of correcting distortion based on the image center calculated by the calibration process, and a display step of displaying the distortion-corrected image A wide-angle image display method comprising: a display process;   2. The wide-angle image display method according to claim 1, wherein, in the imaging step, the calibration chart is a plane on which regularly arranged patterns are printed.   The wide-angle image display method according to claim 1, wherein in the imaging step, the calibration chart is a flat display on which regularly arranged patterns are displayed. An imaging step of capturing a calibration chart by an imaging means having a wide-angle lens and acquiring image data, a calibration chart area detecting step of detecting a calibration chart area on the image, and a circular dot on the captured calibration chart A circular dot detection step for obtaining the center point of the region, an associating step for associating the pixels on the image with the points on the calibration chart, and an error when the fitting is performed on a point group arranged on a straight line on the calibration chart. A calibration mode having a straight line fitting step for selecting a plurality of straight lines, and an image center calculating step in which the intersection of the selected plurality of straight lines is the image center of the omnidirectional image;
An imaging step of acquiring image data by imaging by the imaging means, a distortion correction step of correcting distortion based on the image center calculated in the calibration mode, and a display step of displaying the distortion corrected image A wide-angle image display system.   5. The wide-angle image display system according to claim 4, wherein, in the imaging step, the calibration chart is a plane on which regularly arranged patterns are printed.   5. The wide-angle image display system according to claim 4, wherein, in the imaging step, the calibration chart is a flat display on which regularly arranged patterns are displayed.

Images