JP2887398B2 - Multiple camera correction method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Object of the Invention] (Industrial application field) The present invention relates to a multiple camera correction method for performing geometric correction between cameras when using multiple cameras.

(Prior Art) When capturing an image using a plurality of cameras, it is desired that each camera output the same level at the video signal level and the geometric level. As a conventional camera correction method using such a plurality of cameras, for example, there is a method as shown in a flowchart of FIG. The figure shows a portion related to the correction of the luminance signal levels from a plurality of cameras. A chromaticity pattern for a test is input to each camera input unit (step 41), and the input signal is converted into a digital signal by an AD conversion unit. (Step 42). In the shading unit, the illumination unevenness of the image is reduced (Step 4
3). The signals regulated to the same level are subjected to non-linear conversion by the gamma correction unit in order to enhance the realism of the image (step 44). Then, through the color separation unit,
The multiplexed primary color signals of R, G, and B are separated into R, G, and B (step 45). The above correction of the video signal is performed for each camera by the same processor in a time-division manner.

On the other hand, geometric correction is mechanically performed by an adjusting screw or the like attached to the camera.

(Problems to be Solved by the Invention) When a conventional multiple camera is used, the geometric correction between the cameras is mechanically performed by an adjusting screw or the like attached to the camera. There is a problem that it is difficult to reproduce a uniform image with high precision, which is considerably coarser than one pixel.

Therefore, the present invention provides a multiple camera correction method capable of performing geometric correction between each camera with high accuracy when using multiple cameras and reproducing uniform images with high accuracy. The purpose is to provide.

[Configuration of the Invention] (Means for Solving the Problems) In order to solve the above problems, the present invention provides a multiple camera correction method including geometric correction between cameras when using multiple cameras, A solid having a wire frame structure is input from each of the cameras, and a spatial distortion state of a straight line constituting the solid in the stereoscopic image is detected by image processing, and from each of the cameras according to the detected distortion degree. The gist is that geometric correction between the cameras is performed by correcting the three-dimensional input image.

(Operation) In the above configuration, spatial distortion of a straight line constituting the solid in a three-dimensional image is detected by image processing, and a three-dimensional input image from each camera is corrected in accordance with the detected degree of distortion. Thus, the geometric correction between the cameras is performed with high accuracy. Therefore, it is possible to reproduce a uniform image with high accuracy.

(Embodiment) The multiple camera correction method of this embodiment is basically performed by the following processing. That is, first, a transparent solid body having a lattice pattern is input from a plurality of cameras of the input camera unit, and a luminance signal from each camera is corrected by a video signal level processing unit in a first stage. Next, in the second step, the degree of distortion of the lattice pattern from each camera is corrected by the chemical-chemical distortion correction processing unit, the combined image from each seamless camera is transmitted through the transmission unit, and the combined image is displayed on the display unit. To display.

Hereinafter, the above processing will be described with reference to FIGS. 1 to 7.
This will be described in more detail.

FIG. 1 is a flowchart showing the flow of processing for correcting a luminance signal level and a geometrical deviation between cameras when a plurality of cameras are used. As a landscape to be input to a plurality of cameras, a transparent solid with a colored rectangular lattice pattern is input from a camera input unit (steps 1 and 2). Then, the video signal output from each camera is subjected to a first correction by a video signal level processing unit (step 3). The contents of this correction are performed in substantially the same manner as described with reference to FIG.

Next, as a second correction, the geometric distortion correction processing unit detects a straight line forming a grid by image processing on a three-dimensional and internal grid pattern. Next, an angle between the straight lines is detected. From the input of the rectangular lattice, the degree of distortion of the lattice is known. Based on the detected degree of distortion of each camera, correction is performed on the input image from each camera (step 4). The image corrected by the cameras is transmitted from the transmission unit to the display unit (steps 5 and 6).

FIG. 2 shows a solid 10 input for performing geometric correction on an image from a camera. 3D
Has a wire frame structure so that the lattice pattern inside can be seen. In image processing, each straight line 7, 8, 9, 11, 12, 13, 1 constituting a wire frame in each direction
To make it easy to detect 4, 15, 16 and identify it,
The straight lines in each direction are painted in several colors. The horizontal straight lines 14, 15, and 16 have a hue of 0 to 120 degrees, the vertical straight lines 7, 8, and 9 have a hue of 121 to 239 degrees, and the straight lines 11, 12, and 13 have a hue of 240 to 359 degrees. Are painted in different colors. In addition, in order to make it easy to see the difference between the image input to the camera and the original image, the angle between the straight lines forming the wire frame is set to a right angle in advance.

FIG. 3 shows an example of a typical distortion when a rectangular grid, which is a part of the solid body 10 (hereinafter also referred to as a wire frame, using the same reference numeral), is input from a camera. Each straight line constituting a part 17 of the wire frame before input to the camera is at a right angle as shown in FIG.
However, the image of the wire frame is distorted depending on how each camera is installed. When the camera is displaced in the left-right direction, the image is distorted as shown in FIGS. 4B and 4C, and when the camera is displaced in the up-down direction, the images are shown in FIGS. Distorted like).

FIG. 4 is a diagram for explaining that thinning is performed on a lattice image that is a part of the three-dimensional object 10 and the angle between straight lines is detected. As shown in FIG. 7A, when a rectangular grid image is input, the width of each straight line is not uniform as shown in the enlarged view of FIG. Therefore, as shown in FIG. 3C, each line is thinned by image processing. Such a process is indispensable when detecting the angle between the straight lines.

FIG. 5 is a diagram for describing a method of detecting a spatial distortion amount of each straight line from a wire frame image. To detect a straight line, a hough transform algorithm widely used in the field of image processing is applied. The wireframe image is subjected to the HSV conversion from RGB, and based on the hue information (step 21), the linear components in the vertical, horizontal, and depth directions are detected (step 22). The vertical and horizontal linear components in each depth direction are detected using the hough transform (step 23). The angle formed by the vertical and horizontal straight lines is detected, and the amount of distortion in each direction is detected (step 24).

FIG. 6 is a diagram for explaining a color space used when detecting a colored wire frame. RGB
The wire frame image captured from the camera as the color space information is converted into the HSV color space so that the linear components in the vertical, horizontal, and depth directions can be easily identified. H26, S25, and V27 represent hue, saturation, and lightness, respectively (FIG. 10A). Here, the hue takes a value of 0 to 360 degrees with reference to the red color 28 (FIG. 9B). For example, 6
The colors corresponding to the hue of each 0 degree correspond to red R, yellow Y, green G, cyan C, blue b, and magenta M. When the brightness takes a value from “0” to “1” normalized, the minimum value “0” represents black and the maximum value “1” represents white. Saturation is proportional to the value of lightness. In order to perform extraction of straight lines in each of the vertical, horizontal, and depth directions based on the hue information, the wire frame 10 changes the color of the horizontal straight line from 0 degree.
A color distributed in the hue of 120 degrees 29, the vertical direction is 121 degrees to 240 degrees 30, and the depth direction is 241 degrees to 260 degrees 31 is selected (FIG. 10C).

FIG. 7 is a diagram for explaining the principle of the hough transform used when detecting a straight line. Assuming that the x and y coordinate values of arbitrary points 32, 33, and 34 on the straight line to be detected are on a straight line ρ represented by the following equation expressed in a polar coordinate format, ρ = x · cos θ + y · three straight lines ρ ₁ , ρ ₂ , ρ
₃ can be described.

ρ ₁ = x ₁ · cos θ + y ₁ · sin θ ρ ₂ = x ₂ · cos θ + y ₂ · sin θ ρ ₃ = x ₃ · cos θ + y ₃ · sin θ (Fig. 7 (a)) and draw the graph to get 3
Sinusoidal curves such as 7, 38, and 39 are obtained (FIG. 9B). In this graph, the intersection 40 of the three curves
Represents a straight line to be detected. In this way, straight lines in each direction are detected from the wire frame image captured from the camera.

Through the above processing, the correction of the video signal level and the geometric correction are all performed electronically with high accuracy.

[Effects of the Invention] As described above, according to the present invention, a solid having a wire frame structure is input from each camera, and the spatial distortion of a straight line constituting the solid in an image of the solid is determined by image processing. Detects and corrects the three-dimensional input image from each camera according to the detected degree of distortion to perform geometric correction between each camera. This has the advantage that a uniform image can be reproduced with high accuracy.

[Brief description of the drawings]

1 to 7 are diagrams for explaining an embodiment of a method for correcting a plurality of cameras according to the present invention. FIG. 1 is a flowchart for explaining the overall processing flow, and FIG. 2 is a wire frame structure. FIG. 3 is a diagram showing an example of distortion of a stereoscopic image input to a camera, FIG. 4 is a diagram for explaining thinning processing of a straight line constituting the stereoscopic image, and FIG. 6 is a flowchart for explaining a method of detecting the amount of distortion of a wire frame, FIG. 6 is a diagram for explaining an HSV color space, FIG. 7 is a diagram for explaining the principle of straight line detection, and FIG. 5 is a flowchart for explaining video signal level correction. 10: solid body with a wire frame structure 7, 8, 9, 11, 12, 13, 14, 15, 16: each straight line constituting the solid body.

Claims (1)

(57) [Claims] 1. A multi-camera correction method including a geometric correction between cameras when a plurality of cameras are used, wherein a three-dimensional image having a wire frame structure is input from each of the cameras, and the three-dimensional image is processed by image processing. The spatial correction of the straight line forming the three-dimensional object is detected, and the three-dimensional input image from each of the cameras is corrected in accordance with the detected degree of distortion, thereby performing geometric correction between the cameras. A multiple camera correction method.