JP2025070540A - Image processing system and image processing method - Google Patents

Abstract

The present invention provides a means for enabling highly accurate distance measurement with little parallax error in a stereo camera in which the relative relationship between the cameras is unknown.
[Solution] An image processing system comprising multiple imaging devices, a first screen, a second screen, and an image processing device. The image processing device comprises a position and angle estimator which estimates the position and angle of each of the multiple imaging devices, a parallelized optical axis creator which parallelizes the optical axes of the multiple imaging devices by image processing, a parallelized image creator which creates parallelized images of images captured by the multiple imaging devices, a direction correction processor which orthogonally aligns the images captured by the multiple imaging devices with respect to the second screen, a distance difference corrector which corrects the measured distance from each of the multiple imaging devices to the second screen, a parallax calculator which calculates the parallax of images captured by the multiple imaging devices, and a relative angle correction value calculator which calculates a relative angle error correction value between the multiple imaging devices.
[Selected Figure] Figure 1

Description

The present invention relates to an image processing system and method used to calibrate an in-vehicle stereo camera.

Normally, for in-vehicle stereo cameras, the longer the distance between the left and right cameras, i.e. the longer the baseline length, the more advantageous it is for distance measurement. Furthermore, when in-vehicle stereo cameras are used as sensors for autonomous driving, the optical axes of the two cameras must be aligned with high precision. If the two cameras are housed in the same small housing, the rigidity of the housing allows the optical axes to be precisely kept in the same direction, but to achieve a long baseline length, the two cameras must be housed in separate housings, making it difficult to physically align the optical axes. To address this issue, Patent Document 1 uses a method in which the optical axis misalignment of the camera is determined from the image captured by the camera and the camera is physically moved. Furthermore, Patent Document 2 detects the misalignment of the optical axis by capturing images of the same pattern with the camera at intervals that match the baseline length.

JP 2018-031733 A International Publication No. 2019/087253

However, providing an on-board stereo camera with a power mechanism for moving the cameras as shown in Patent Document 1 leads to an increase in cost. In addition, the method shown in Patent Document 2 requires two charts to be placed at a distance equal to the baseline length, which means that it cannot be used when the exact baseline length is unknown, and the two cameras must be almost directly facing the charts. Thus, there is a demand for technology that can correct the optical axis misalignment for on-board stereo cameras with unknown baseline lengths to achieve highly accurate distance measurements.

The present invention was made in consideration of the above problems, and aims to provide a means for enabling highly accurate distance measurement with little parallax error in a stereo camera where the relative relationship between the cameras is unknown.

In order to solve the above problem, the image processing system of the present invention is an image processing system comprising a plurality of imaging devices, a first screen on which a repeating pattern is drawn and which is positioned at a first distance from the plurality of imaging devices, a second screen on which a repeating pattern and a non-repeating pattern are drawn and which is positioned at a second distance from the plurality of imaging devices, and an image processing device, in which the image processing device includes a position and angle estimation unit that estimates the position and angle of each of the plurality of imaging devices based on the imaging results of the repeating pattern captured by the plurality of imaging devices, a parallelization optical axis creation unit that parallelizes the optical axes of the plurality of imaging devices by image processing using the estimation results of the position and angle estimation unit, and a parallelization optical axis creation unit that performs image processing based on the processing results of the parallelization optical axis creation unit. The system includes a parallelized image creation unit that creates parallelized images of images captured by the multiple image capture devices, a parallelization processing unit that uses the images created by the parallelized image creation unit to parallelize the images captured by the multiple image capture devices with respect to the second screen, a distance difference correction unit that corrects the measured value of the distance from each of the multiple image capture devices to the second screen by adjusting the magnification of the parallelized image using the estimation result of the position angle estimation unit, a parallax calculation unit that calculates the parallax of the images captured by the multiple image capture devices based on the imaging result of the non-repeating pattern that reflects the processing result of the distance difference correction unit, and a relative angle correction value calculation unit that calculates a relative angle error correction value between the multiple image capture devices based on the calculation result of the parallax calculation unit and the second distance.

By using the present invention, it is possible to provide a means for enabling highly accurate distance measurement with little parallax error in a stereo camera in which the relative relationship between the cameras is unknown.
Further features related to the present invention will become apparent from the description of the present specification and the accompanying drawings. Furthermore, the objects, configurations and effects other than those described above will become apparent from the following description of the embodiments.

FIG. 1 is a block diagram showing a functional configuration of an image processing system according to an embodiment of the present invention. 4 is a flowchart showing a process executed by an image processing system according to an embodiment of the present invention. FIG. 2 shows a screen arrangement according to one embodiment of the present invention. 1 is an example of a chart arrangement used in the present invention. 13 is another example of a chart arrangement used in the present invention.

Below, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the overall functional configuration of an image processing system according to an embodiment of the present invention. The image processing system includes vehicle-mounted cameras 101 and 102, an image processing device 100, and multiple screens. The multiple screens specifically refer to the first screens 302 and 303, which are arranged at a first distance from the camera units 304 and 305 and on which a repeating pattern is drawn, and the second screen 301, which is arranged at a second distance from the camera units 304 and 305 that is farther than the first distance and on which a repeating pattern and a non-repeating pattern are drawn, as shown in FIG. 3. The image processing device 100 is composed of a computer equipped with a CPU (Central Processing Unit) and memory, and has the following functional units:

That is, the image processing device 100 includes distortion removal units 103 and 104 that correct the lens distortion of the wide-angle camera, repeating pattern detection units 105 and 106 that detect a repeating pattern used for calibration from the image after distortion removal, camera position angle estimation units 107 and 108 (corresponding to the position angle estimation unit in the claims) that estimate the angle and position of the camera from the repeating pattern, a relative parameter calculation unit 109 that calculates the relative position and relative angle of the two cameras, a parallelization optical axis creation unit 110 that determines the parallelization optical axis from the relative position and relative angle and creates a rotation matrix for parallelization, and a parallelization unit 120 that performs parallelization on the distortion-removed image. It is equipped with parallelized image creation units 111 and 112 that perform parallelization processing to create parallelized images, orientation processing units 113 and 114 that perform processing to orient the parallelized images relative to the screen, distance difference correction units 115 and 116 that adjust the distance from the two cameras to the screen, parallax calculation unit 117 that calculates the parallax of the non-repeating pattern from the two orientated images, relative angle correction value calculation unit 118 that calculates a correction value for the relative angle error between the imaging devices from the parallax, correction value reflection unit 119 that corrects the relative angle using the correction value, and an output terminal 120 that outputs the corrected relative parameters as calibration values.

Next, the operation of each part will be described. An in-vehicle stereo camera has two or more camera units, but in this embodiment, two cameras will be described. The two cameras are placed in different positions, usually on the left and right. In addition, the distortion removal unit 103 to the output terminal 120, which are functional units that input the image capture results of the cameras and output the calculation results, may be placed independently as separate devices, or may be placed as a package in which some or all of them are contained in a single housing. In other words, the image processing system according to this embodiment constitutes a single device as a whole in this embodiment, but it may also have a configuration in which multiple devices are combined. In addition, in order to achieve a long baseline length, the in-vehicle stereo camera may be mounted on the vehicle with the two cameras housed in separate housings, and this is referred to as a "separate camera" in this embodiment.

As described below, the two vehicle-mounted cameras 101 and 102 capture images containing repeating and non-repeating patterns, but the resulting images are output as images containing distortion caused by the wide-angle lens. Since this distortion is specific to the camera unit, it is converted into a distortion-free image by distortion removal units 103 and 104 using a correction table or the like.

The images with distortion removed are sent to repeating pattern detection units 105 and 106, which detect a known repeating pattern in each image. This repeating pattern has known markers arranged repeatedly, and the position of each marker on the image is detected. As described below, the repeating patterns are arranged at multiple distances, and at least one repeating pattern is drawn on the same plane as a non-repeating pattern.

Images in which repeating patterns arranged at multiple distances are detected are sent to camera position angle estimation units 107 and 108. Since the positions of the markers in the image space are known, if there are three or more markers, the three-dimensional position and three angles of the camera can be calculated by camera position angle estimation units 107 and 108. The calculation method can be, for example, a method in which a design position and angle are given as initial values, and the projected position on the imaging surface is calculated using these parameters, and the difference from the actual marker detection position is used as an error function, and the parameters are modified to converge in the direction that reduces the error function, but the method is not limited to this.

The positions and angles of the two cameras estimated by the camera position angle estimation units 107 and 108 are sent to a relative parameter calculation unit 109. The relative parameter calculation unit 109 calculates the difference between the positions and angles of the two cameras and outputs it as a relative parameter to the parallelization optical axis creation unit 110.

The optical axes of the two cameras 101 and 102 are physically oriented in different directions, and the optical axis directions must be aligned to correctly calculate parallax. Physically changing the orientation of the cameras requires highly accurate orientation control, which is not realistic for an in-vehicle stereo camera, so a method is used in which images with aligned orientations for the left and right cameras are created by image processing without changing the physical orientation of the cameras. This processing is called parallelization processing, and the parallelized optical axis creation unit 110 creates the orientation of the virtual optical axes after parallelization processing.

The direction of the parallelization optical axis can be determined by using one of the left and right cameras as a reference camera and aligning the other camera, or by taking the middle point between the directions of the two optical axes. After parallelization processing, a rotation matrix is created from the angle difference between the optical axis of the original camera and the virtual parallelization optical axis. This rotation matrix is sent to parallelization image creation units 111 and 112. The parallelization image creation units 111 and 112 apply the rotation matrix sent from the parallelization optical axis creation unit 110 to the distortion-free images output from the distortion removal units 103 and 104 to create images aligned with the direction of the virtual parallelization optical axis. The created images are sent to the orientation processing units 113 and 114.

The orientation processing units 113 and 114 create a rotation matrix from the difference between the virtual parallel optical axis and the orientation of the screen, and convert the images according to the rotation matrix to convert the images captured by the two cameras into images that are directly facing the screen and have the same orientation for the left and right cameras. At this time, the images captured by the two cameras are directly facing the screen on which the non-repeating pattern and the repeating pattern are drawn on the same plane. In particular, in the case of a separate type camera, unlike a case in which two cameras are housed in one housing, it is difficult to physically guarantee the relative relationship between the two cameras, so there may be a deviation in the relative relationship (relative angle, etc.) between the two cameras. The orientation processing by the orientation processing units 113 and 114 makes it possible to orient the two cameras to the screen on which the non-repeating pattern is drawn, even if the two cameras are not directly facing the screen, and enables the parallax calculation unit 117 to calculate the parallax with high accuracy in the subsequent stage. This enables the relative angle correction value calculation unit 118 to calculate a highly accurate relative angle correction value. Note that although the parallelization process and the rectification process are described here as separate processes, since both are rotation matrix processes, it is also possible to create a rotation matrix that combines both processes and perform image transformation at once.

Since the distances from the two cameras to the screen are different for images that have been subjected to the normalization process, the images need to be enlarged or reduced. This process is carried out by the distance difference correction units 115 and 116. Specifically, the distance difference correction units 115 and 116 match the magnification of one of the normalized images to that of the other image, thereby making both images equal in size, which is essentially the same process as adjusting the distance between the two cameras and the screen.

The image after the distance difference correction process is sent to the parallax calculation unit 117. The parallax calculation unit 117 detects parallax using the non-repeating pattern. Here, parallax is an index that indicates the difference in how the imaged object looks from the left and right cameras, and methods for calculating it are known, such as the SGM (Semi Global Matching) method and the SAD (Sum of Absolute Difference) method. These calculation methods are already known, so a detailed explanation will be omitted. Since the non-repeating pattern is a known pattern, the known pattern is found from the images obtained from the left and right cameras by correlation processing, and the parallax is calculated by finding the difference in their positions. In principle, the theoretical value of this parallax is determined by the distance from the camera to the screen of the non-repeating pattern, but the actually measured parallax value has a difference due to the angle estimation error. The relative angle of the two cameras can be corrected by converting this parallax difference into an angle. This calculation is performed by the relative angle correction value calculation unit 118.

This correction value is sent to the correction value reflection unit 119, and the relative parameter reflecting the correction value calculated by the relative angle correction value calculation unit 118 is output from the output terminal 120 as the final output. The angle reflected as the correction value is usually a small angle, but if this correction value is large, the reliability of the value calculated by the relative parameter calculation unit 109 is considered to be low, so if the correction value exceeds a predetermined threshold, it is possible to output an error without outputting the corrected value. In this case, it is possible to start abnormal processing, such as starting over from image acquisition, or using it as a trigger for a process such as reattaching the camera.

The value corrected by the correction value reflection unit 119 can be output directly from the output terminal 120, but it is also possible to verify the corrected value again using a non-repeated pattern. The operation when performing repeated verification will be explained according to the process flow shown in Figure 2. Note that in this flow, two cameras are assumed to be arranged in a left-right direction facing the screen that is the subject of the image capture.

First, the images captured by both the left and right cameras are input to the distortion removal unit 103, and an image with distortion removed is generated. Then, step 201 is an initial estimation process for the camera position and camera angle using a repeating pattern drawn on a chart placed at multiple distances, which will be described later. This corresponds to the processing in the repeating pattern detection units 105 and 106, the camera position angle estimation units 107 and 108, and the relative parameter calculation unit 109. After executing the processing in step 201, the process proceeds to step 202.

In step 202, the left and right camera images are parallelized based on the results obtained in step 201. This corresponds to the processing performed by the parallelization optical axis creation unit 110 and parallelized image creation units 111 and 112. In step 203, which follows, a straightening process is performed on the screen on which both repeating and non-repeating patterns are drawn, as described below. This corresponds to the processing performed by the straightening units 113 and 114.

In step 204, the distance from the camera position determined in step 201 to the screen on which both the repeating pattern and the non-repeating pattern described below are drawn is calculated, and image conversion is performed to eliminate the distance difference between the left and right cameras. This corresponds to the processing of distance difference correction units 115 and 116. In the subsequent processing of step 205, parallax calculation unit 117 uses the image that has been corrected for the distance difference to perform parallax calculation using the non-repeating pattern that has been subjected to the orientation processing in step 203.

Since the theoretical value of parallax is determined by the distance from the camera to the non-repeating pattern that has been subjected to the orientation process in step 203, the theoretical value is calculated from the camera position found in step 201. This theoretical value and the parallax value actually obtained from the image will match if the camera position and angle found in step 201 are correct, but if an error exists, a difference will occur accordingly. This difference becomes the correction value for the relative angle, and high accuracy can be achieved because there are only two parameters to be corrected, the camera position and angle. The calculation of this correction value is performed by relative angle correction value calculation unit 118.

In step 206, the difference is judged, and if the difference is less than a predetermined value, it is determined that the camera position and angle found in step 201 are correct. In the next step 207, the correction value reflection unit 119 reflects the relative angle corresponding to the difference in the angle found in step 201, and the process ends. This reflected value becomes the final output.

If the difference is equal to or greater than the specified value in step 206, the correction value reflection unit 119 counts the number of repetitions up to that point in step 208. If the number of repetitions is equal to or greater than a predetermined specified value, it is determined that the error that has occurred up to that point has become so large that it can no longer be corrected, and the process ends as a failure. If the number of repetitions is less than the specified value, the process proceeds to step 209, where the correction value reflection unit 119 reflects the difference calculated in step 205 in the camera angle. The process from step 202 to step 205 is performed again for this new camera angle, and the difference between the theoretical value and the parallax value obtained from the image is calculated in step 206. If the difference is again equal to or greater than the specified value, the difference is further reflected and the process from step 202 to step 205 is repeated. If the difference does not fall below the specified value even after a certain number of repetitions, the process ends as a failure based on the judgment in step 208.

The flow described above uses repeating patterns arranged at multiple distances to make an initial estimate of the camera position angle, then the captured image is aligned with a panel on which both repeating and non-repeating patterns are drawn to correct the distance difference between the left and right cameras, thereby correcting the relative angle error between the cameras. This makes it possible to perform highly accurate parallax correction and perform accurate calibration, even in an in-vehicle stereo camera system where the relative relationship between the left and right cameras is unknown, such as when two cameras are mounted on a vehicle in separate housings, such as a separate camera.

Figure 3 shows an example of a screen layout used in one embodiment of the present invention. Figure 3 is a diagram of the layout as seen from above. In this embodiment, as mentioned above, 301 is a second screen on which a repeating pattern and a non-repeating pattern are arranged, 302 and 303 are first screens on which only a repeating pattern is arranged, and 304 and 305 are camera units.

Examples of repeating and non-repeating patterns arranged on the second screen 301 are shown in Fig. 4 and Fig. 5. First, in Fig. 4, 401 and 402 are repeating patterns, and 403 is a non-repeating pattern. Also, in Fig. 5, 501 and 502 are non-repeating patterns, and 503 is a repeating pattern. Since the non-repeating pattern needs to be on the same plane as one of the repeating patterns, one of the charts arranged in Fig. 3 is a chart with repeating and non-repeating patterns drawn on it. By drawing the non-repeating and repeating patterns on the same plane on the second screen 301, it is possible to perform a normalization process on the screen with the repeating pattern and a normalization process on the non-repeating pattern at the same time, and the accuracy of the parallax calculation using the non-repeating pattern by the parallax calculation unit 117 can be improved. As a result, the relative angle error between the cameras can be calculated with high accuracy. Furthermore, in this embodiment, the second screen 301 with the non-repeating pattern drawn on it is placed farther from the camera than the first screens 302 and 303 with only the repeating pattern drawn on them. This is because the non-repeating pattern is used to measure the camera's disparity, and the greater the distance between the camera and the non-repeating pattern, the more accurate the disparity measurement.

The repeating pattern in Fig. 4 is a grid of circles, and in Fig. 5 is a chessboard-type grid pattern, but it is not limited to these examples as long as the pattern is easy to detect the characteristic points and is arranged in a regular pattern so that detection errors on the image are unlikely to occur. The non-repeating patterns in both Fig. 4 and Fig. 5 are non-repeating patterns, but there is no problem as long as the same pattern is unlikely to occur in multiple places. Fig. 4 is an example of two repeating patterns and one non-repeating pattern, and Fig. 5 is an example of one repeating pattern and two non-repeating patterns, but the number of patterns is not limited to these. Furthermore, in Fig. 3, chart boards 302 and 303 are composed only of repeating patterns, so they are the same as the repeating patterns shown in Figs. 4 and 5. If the design of the repeating patterns is the same for all charts, the same detection method can be used, so it is preferable to make them the same, but it is also possible to use different designs.

In Figure 3, all screens are set up vertically on the floor so that the camera faces them almost directly, but the first screen (302 and 303 in Figure 3), which is made up only of a repeating pattern, does not need to be set up vertically as it does not need to be directly facing the camera. For example, it is also possible to attach it to the floor or ceiling. If there is not enough space horizontally, it is also possible to attach it to the wall.

The above-described embodiment of the present invention provides the following effects:

(1) The image processing system of the present invention is an image processing system comprising a plurality of imaging devices, a first screen on which a first chart, which is a repeating pattern, is drawn and which is positioned at a first distance from the plurality of imaging devices, a second screen on which a repeating pattern and a non-repeating pattern are drawn and which is positioned at a second distance from the plurality of imaging devices, and an image processing device, in which the image processing device includes a position and angle estimation unit that estimates the position and angle of each of the plurality of imaging devices based on the imaging results of the repeating pattern captured by the plurality of imaging devices, a parallelization optical axis creation unit that parallelizes the optical axes of the plurality of imaging devices by image processing using the estimation results of the position and angle estimation unit, and a parallelization optical axis creation unit that performs image processing based on the processing results of the parallelization optical axis creation unit. The system includes a parallelized image creation unit that creates parallelized images of images captured by the multiple image capture devices, a parallelization processing unit that uses the images created by the parallelized image creation unit to parallelize the images captured by the multiple image capture devices with respect to the second screen, a distance difference correction unit that corrects the measured value of the distance from each of the multiple image capture devices to the second screen by adjusting the magnification of the parallelized image using the estimation result of the position angle estimation unit, a parallax calculation unit that calculates the parallax of the images captured by the multiple image capture devices based on the imaging result of the non-repeating pattern that reflects the processing result of the distance difference correction unit, and a relative angle correction value calculation unit that calculates a relative angle error correction value between the multiple image capture devices based on the calculation result of the parallax calculation unit and the second distance.

The above configuration provides a means for enabling highly accurate distance measurement with minimal parallax error in a stereo camera where the relative relationship between the cameras is unknown.

(2) The second distance is greater than the first distance. By placing the second chart at a distance greater than the first distance, it becomes possible to measure the parallax between the imaging devices more accurately, and as a result, it becomes possible to correct the relative error between the imaging devices more accurately.

(3) If the relative angle error correction value calculated by the relative angle correction value calculation unit is greater than a predetermined threshold, the image processing device issues an error. This makes it possible to decide whether to discard the correction value and start over from image acquisition.

(4) When the relative angle error correction value calculated by the relative angle correction value calculation unit is greater than a predetermined threshold, the image processing device reflects the relative angle error correction value in the multiple image capture devices and then executes image processing again. This makes it possible to repeatedly fine-tune the image capture devices and improve the accuracy of the correction.

(5) The image processing method according to the present invention estimates the position and angle of each of the multiple imaging devices based on the results of imaging a repeating pattern depicted on a first screen and a second screen located a first distance and a second distance away from the multiple imaging devices, collimates the optical axes of the multiple imaging devices using the estimation results, orients the images captured by the multiple imaging devices relative to the second screen using the results of collimation, corrects the measured value of the distance from each of the multiple imaging devices to the second screen by adjusting the magnification of the orientated image using the estimation results, calculates the parallax of the images captured by the multiple imaging devices based on the results of imaging a non-repeating pattern depicted on the second screen with the distance measurement corrected, and calculates a correction value for the relative angle error between the multiple imaging devices based on the calculated parallax and the second distance. This is expected to have the same effect as (1). The method according to the present invention also has the same configuration and effect as (2) to (4).

The present invention is not limited to the above-mentioned examples, and various modifications are possible. For example, the above-mentioned examples have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to an embodiment that includes all of the configurations described. It is also possible to replace part of the configuration of one example with the configuration of another example. It is also possible to add the configuration of another example to the configuration of one example. It is also possible to delete part of the configuration of each example, or to add or replace other configurations.

101, 102 Camera (imaging device), 107, 108 Camera position angle estimation unit (position angle estimation unit), 110 Parallel optical axis creation unit, 111, 112 Parallel image creation unit, 113, 114 Orientation processing unit, 115, 116 Distance difference correction unit, 117 Parallax calculation unit, 118 Relative angle correction value calculation unit, 301 Second screen 302, 303 First screen

Claims (8)

A plurality of imaging devices;
a first screen having a repeating pattern and positioned a first distance from the plurality of image capture devices; and a second screen having a repeating pattern and a non-repeating pattern and positioned a second distance from the plurality of image capture devices.
An image processing system comprising:
The image processing device includes:
a position and angle estimation unit that estimates a position and an angle of each of the plurality of imaging devices based on an imaging result of the repeating pattern captured by the plurality of imaging devices;
a parallelized optical axis creation unit that parallelizes the optical axes of the plurality of image capture devices by image processing using the estimation result of the position angle estimation unit;
a parallelized image creation unit that creates a parallelized image by parallelizing the images captured by the plurality of image capture devices based on a processing result of the parallelization optical axis creation unit;
a rectification unit which rectifies the images captured by the plurality of image capture devices relative to the second screen using the images created by the rectification image creation unit;
a distance difference correction section which corrects the measured distance from each of the plurality of image capture devices to the second screen by adjusting the magnification of the orthogonalized image using the result of estimation by the position and angle estimator;
a parallax calculation unit that calculates parallax of images captured by the plurality of imaging devices based on an imaging result of the non-repeated pattern reflecting a processing result of the distance difference correction unit;
a relative angle correction value calculation unit that calculates a relative angle error correction value between the plurality of image capture devices based on a calculation result of the parallax calculation unit and the second distance.
1. An image processing system comprising: 2. The image processing system according to claim 1,
the second distance is greater than the first distance;
1. An image processing system comprising: 2. The image processing system according to claim 1,
The image processing device issues an error when the relative angle error correction value calculated by the relative angle correction value calculation unit is greater than a predetermined threshold value.
1. An image processing system comprising: 2. The image processing system according to claim 1,
when the relative angle error correction value calculated by the relative angle correction value calculation unit is greater than a predetermined threshold value, the image processing device reflects the relative angle error correction value in the plurality of image capturing devices and then performs image processing again.
1. An image processing system comprising: 1. An image processing method performed by an image processing system, comprising:
estimating the position and angle of each of the plurality of image capturing devices based on the captured images of a repeating pattern depicted on a first screen and a second screen that are positioned at a first distance and a second distance from the plurality of image capturing devices;
Using the result of the estimation, the optical axes of the plurality of imaging devices are collimated;
Using the result of the parallelization, the images captured by the plurality of image capturing devices are aligned in a normal direction relative to the second screen;
using the result of said estimation to correct a measurement of the distance from each of said plurality of image capture devices to said second screen by adjusting a magnification of said orthogonalized image;
calculating a parallax of the images captured by the plurality of image capture devices based on the image capture results of the non-repeating pattern drawn on the second screen after the distance measurement values have been corrected;
calculating a relative angle error correction value between the plurality of image capturing devices based on the calculated parallax and the second distance;
13. An image processing method comprising: 6. The image processing method according to claim 5,
The second distance is greater than the first distance.
13. An image processing method comprising: 6. The image processing method according to claim 5,
if the calculated relative angle error correction value is greater than a predetermined threshold, issue an error.
13. An image processing method comprising: 6. The image processing method according to claim 5,
if the calculated relative angle error correction value is greater than a predetermined threshold, the relative angle error correction value is reflected in the plurality of image capture devices, and then image processing is performed again.
13. An image processing method comprising:

Images