JP2002135644A - Method for calculating and detecting positional deviation of camera, and image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem of the positional deviation of a camera being detected erroneously as a result of calculating the positional deviation of the camera erroneously, because the matching ratio of another similar pattern sometimes becomes higher than that of an original pattern, due to mixing of noise into an input picture or the change of illumination, when the pattern similar to the matching pattern of a reference image exists in the input image. SOLUTION: Pattern matching with the input image is performed with plural matching patterns, to obtain a moving image from a positional deviation between the positional deviation candidate area of the input picture high in the matching ratio and the area of the matching patterns to re-conduct pattern matching at the same position of the area of the plural matching patterns with the moving image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a method of calculating a camera position shift amount from a plurality of images captured by a camera and a method of detecting a camera position shift.

[0002]

2. Description of the Related Art Conventionally, a method of calculating a camera position shift amount and a method of detecting a camera position shift will be described with reference to FIG. 4 showing a camera position shift detection processing flow, FIG. 5 showing an operation of FIG. FIG. 6 shows a block diagram of an image processing system in the case of a John camera.

First, the configuration of an image processing system will be described with reference to FIG. The image processing system in FIG. 6 includes a television camera 101, an image processing device 102, an alarm display device 114, and a video monitor 115.

[0004] The image processing apparatus 102 converts an image signal captured by the television camera 101 into an A / D (Analog to Digit) signal.
al) Image input I / F (Interfac
e, hereinafter referred to as I / F) 103; an image memory 104 used for calculation between images and storing the images; a CPU 108 for calculating and detecting a camera position shift amount using the images; a work memory 110, a program memory 116 as a storage medium storing a processing program for calculating and detecting the camera position shift amount,
/ A (Digital to Analog, hereinafter referred to as D / A) conversion, an image output I / F 112 for generating a video signal, a system bus 113, and an output for outputting an alarm when a positional displacement of the television camera 101 is detected. It is composed of an I / F 109.

[0005] An image memory 104 stores a reference image memory 105 for storing a reference image at a preset camera position, an input image memory 106 for storing an image obtained by the image input I / F 103, and processing used for calculation between images. Memory 10
7.

The image processing device 102 has an image input I / F 1
03 of the video signal of the television camera 101 using A
/ D conversion is performed to obtain an image. The image obtained by the image input I / F 103 is stored in the input image memory 106 in the image memory 104. The CPU 108 calculates a camera position shift amount and detects a camera position shift by using the work memory 110 and the image memory 104 according to a processing program stored in the program memory 116. The image output I / F 112 performs D / A conversion of an image stored in the processing memory 107 in the image memory 104 to generate a video signal. The video monitor 115 receives a video signal obtained by the image output I / F 112 and displays an image. The system bus 113 is a bus that transfers various information between the CPU 108, the image memory 104, and each of the I / Fs. The output I / F 109 outputs an alarm to the alarm display monitor 114 upon receiving alarm information from the processing result of the CPU 108 indicating that the camera position shift has been detected.

Next, a description will be given of a conventional technique for calculating the amount of camera displacement and detecting the camera displacement.

In a reference image capturing step 201, a video signal picked up by the television
After the image is stored in the input image memory 106 via the / F 103 and copied to the reference image memory 105, the reference image 301 is obtained. This reference image 301 becomes a reference image for processing of detecting a camera position shift amount and a camera position shift with respect to an input image captured later.

In an input image capturing step 202, a video signal picked up by the television camera 101 is
The image is stored in the input image memory 106 as an image via the / F 103 and an input image 302 is obtained. This input image 302
Is an image that is sequentially updated based on a video signal from the television camera 101.

In a displacement position calculating step 401, a matching pattern P1 set in advance from the reference image 301 is set.
And pattern matching is performed within the search range 303 of the input image 302, and an area having the highest matching rate as shown in the position shift candidate position image 304 is set as a position shift candidate area P1 ', and the position coordinates (P1') of that area are set.
X, P1′Y) is calculated. Hereinafter, assuming that the coordinates of each axis of an arbitrary position coordinate in an orthogonal coordinate system in which the horizontal direction is the X axis and the vertical direction is the Y axis are x and y, the position coordinates are represented by (x, y).

In the displacement amount calculating step 402, the position coordinates (P1X, P1X) of the area of the matching pattern P1 are determined.
Y) and the position coordinates (P
1′X, P1′Y), using the equations (1) and (2), the X-axis direction and the Y-axis
The amount of displacement ΔP1 in the axial direction is calculated as ΔP1X and ΔP1Y.

The displacement amount ΔP1X = the position X coordinate P1′X of the displacement candidate area P1′−the position X coordinate P 1X of the area of the matching pattern P1... (1) The displacement amount ΔP1Y = the displacement candidate area P1 ′ Y coordinate P1′Y−Position Y coordinate P 1Y of the area of the matching pattern P1... (2) In the positional shift amount determining step 214, the positional shift amounts ΔP1X and ΔP1Y obtained in the positional shift amount calculating step 402 are calculated. It is determined whether the value is equal to or more than a predetermined value. If the value is equal to or more than the predetermined value, the process returns to the input image capturing step 202 after performing the alarm processing step 215. Continue processing. In the alarm processing step 215, an alarm is output to the alarm display monitor 114 via the output I / F 109.

[0013]

In the above prior art,
If a similar pattern to the matching pattern of the reference image exists in the input image, the matching rate is higher in the area of the other similar pattern than in the area of the original pattern due to noise contamination and illuminance change in the input image. There is a problem that the misalignment of the camera is incorrectly calculated and the misalignment of the camera is erroneously detected.

[0014]

According to the present invention, in order to solve such a problem, a different matching pattern Pm from a reference image (m and M are positive integers, 2 ≦ M, m = 1,
2,..., M) M preset and matching patterns Pm
, The input image is searched for, and the displacement candidate area Pm′n (n, N is a positive integer and 1 ≦ N, n
= 1, 2,..., N), and a maximum of M × N position deviation candidate areas Pm′n (Pm′n) are extracted.
X, Pm'nY) and positional deviation candidate area information composed of the position coordinates (PmX, PmY) of the area of the matching pattern Pm are extracted, and the positional deviation candidate area information is rearranged in descending order of the matching ratio, and the higher the matching ratio, Of the area of the matching pattern Pm (PmX, Pm
Y) and the position coordinates (Pm'n) of the position deviation candidate area
X, Pm'nY) to calculate a position shift amount ΔPm, n, and match an input image with a moving image obtained by moving the input image according to the position shift amount ΔPm, n at the same position as the area of the matching pattern Pm. It is determined whether or not the pattern Pm matches, and a maximum of M × N pieces of the determination are obtained from the maximum M × N pieces of positional deviation candidate area information, so that a pattern similar to the matching pattern of the reference image exists in the input image. Even if you do, calculate the amount of camera displacement correctly,
It is intended to detect a camera displacement properly.

[0015]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 shows a detection processing flow, FIG. 2 shows an operation, and an image processing system according to one embodiment of the present invention for calculating a camera displacement and detecting a camera displacement. 3 will be described.

Reference numeral 111 in FIG. 3 denotes a program memory as a critical medium which stores a processing program of the method for calculating the amount of camera displacement and detecting the camera displacement according to the present invention. 3 having the same reference numerals as those in FIG. 6 have the same functions, and the description thereof is omitted here.

In the following, the matching pattern Pm is represented by M =
It is assumed that the number of displacement candidate areas Pm'n is set to N = 5 at the maximum in advance.

In the reference image capturing step 201, the video signal picked up by the television camera 101
After the image is stored in the input image memory 106 via the / F 103 and copied to the reference image memory 105, the reference image 301 is obtained. The reference image 301 is used as a reference image for detecting a camera position shift with respect to an input image captured later.

In the input image capturing step 202, the video signal picked up by the television camera 101 is
The image is stored in the input image memory 106 as an image via the / F 103 and an input image 302 is obtained. This input image 302
Is an image that is sequentially updated based on a video signal from the television camera 101.

Calculating candidate position 1 for positional deviation 203
Then, a predetermined matching pattern P1 is cut out from the reference image 301, and the input image 3
02, and a position shift candidate area P1'1 and its position coordinates (P1'1X, P1'1Y) and a position shift candidate area P1'2 and its position coordinates as shown in a position shift candidate position image 304. (P1'2X, P
1′2Y) is calculated.

Further, a position shift candidate position 2 calculation step 204 and a position shift candidate position 3 calculation step 205
Then, as shown in the displacement candidate position image 304 using the matching patterns P2 and P3, the displacement candidate area P2′1 and its position coordinates (P2′1X, P2′1Y) and the displacement candidate area P3′1 and its Position coordinates (P
3′1X, P3′1Y).

Relocation candidate position rearranging step 206
Then, the displacement candidate areas P1′1, P1′2, P2′1, P2′1, P2′1 obtained in the displacement candidate position 1 calculation step 203, the displacement candidate position 2 calculation step 204, and the displacement candidate position 3 calculation step 205 are obtained. And position coordinates (P1′1X, P1′1Y) of (P1′1Y), (P3′1)
1'2X, P1'2Y), (P2'1X, P2'1
Y) and (P3′1X, P3′1Y) are rearranged in descending order of the matching ratio, and the displacement candidate area Pm′n
Is obtained from the position coordinates (Pm'nX, Pm'nY) and the position coordinates (PmX, PnY) of the area of the matching pattern Pm.

In the processing continuation determination step 207, it is determined whether or not all of the positional deviation candidate area information has been processed, or whether or not the positional deviation amount calculation has been completed. The shift amount ΔPm, n calculation processing (processing steps 208 to 213 described later) is continued.
If this is the case, a displacement amount determination step 214 is performed.

In the input image moving step 208, the position coordinates (Pm'nX, Pm'n) of the position error candidate area Pm'n obtained in the error position rearranging step 206 are obtained.
Y) and the position coordinates (P) of the area of the matching pattern Pm.
mX, PnY), the position shift amount ΔPm, n is calculated as shown in a position shift amount calculation image 305, and the position shift amount ΔP
A moving image 306 is obtained by moving the input image 302 by m and n.

Position confirmation 1 In step 209, the reference image 3
01, a preset matching pattern P1 is cut out, and pattern matching with the moving image 306 is performed at the same position as the matching pattern P1 to obtain the presence or absence of the matching.

Similarly, in the position confirmation 2 step 210 and the position confirmation 3 step 211, the presence or absence of matching in the respective matching patterns P2 and P3 is obtained.

In the matching determination step 212, it is determined whether or not there is a matching obtained in the position confirmation 1 step 209, the position confirmation 2 step 210, and the position confirmation 3 step 211. If there is a matching in all the position confirmation steps, a position shift amount calculation completion step 213 is performed.
If there is no matching even in one position confirmation step,
The process returns to the processing continuation determination step 207.

In the position shift amount calculation completion step 213, a position shift amount calculation completion flag is set, and the process returns to the processing continuation determination step 207.

In the displacement amount determination step 214, the displacement amount ΔPm, n obtained in the input image moving step 208
Is greater than or equal to a predetermined value. If it is greater than or equal to the predetermined value, an alarm processing step 215 is performed, and the process returns to the input image capturing step 202. If the value is equal to or smaller than the predetermined value, the process returns to the input image capturing step 202.

In the alarm processing step 215, the output I /
An alarm is output to the alarm display monitor 114 via F109.

In the following, the position coordinates (Pm'nX, Pm'n) of the displacement candidate area Pm'n in the displacement candidate area information obtained in the displacement candidate position rearranging step 206 are described.
m′nY) is (P1′1X,
(P1'1Y), (P1'2X, P1'2Y), (P2 '
1X, P2'1Y) and (P3'1X, P3'1
Processing continuation determination step 2 based on the example rearranged as Y)
The flow from 07 to the position shift amount calculation completion step 213 will be described.

Since the condition of the processing continuation judging step 207 is not satisfied, the positional deviation amount calculating processing (step 208)
To 213).

In the input image moving step 208, as shown in the displacement amount calculation image 305, the position information (Pm'nX, Pm) of the displacement candidate area Pm'n = P1'1 to be processed.
m′nY) = (P1′1X, P1′1Y) and the position coordinates (PmX, Pm) of the area of the matching pattern Pm = P1
Y) = (P1X, P1Y) and the displacement ΔPm, n
(Δm, nX, Δm, nY) = ΔP1, 1 (Δ1, 1
X, Δ1, 1Y) are calculated from equations (3) and (4).
The moving image 306 is obtained by moving the input image 302 by the calculated position shift amount ΔPm, n.
Here, the moving image 306 is obtained by moving the input image 302 rightward by ΔP1,1.

The displacement amount Δm, nX = the position X coordinate Pm′n X of the displacement candidate area Pm′n−the position X coordinate PmX of the area of the matching pattern Pm (3) the displacement amount Δm, nY = position The position Y coordinate Pm'nY of the displacement candidate area Pm'nY-the position Y coordinate PmY of the area of the matching pattern Pm (4) Position confirmation 1 In step 209, the matching pattern P1 set in advance from the reference image The clipping and pattern matching with the moving image 306 are performed at the same position as the matching pattern, and a result with matching is obtained.

Position confirmation 2 In step 210, reference image 3
01, a predetermined matching pattern P2 is cut out, and the moving image 306 is set at the same position as the matching pattern.
Is performed, and a result without matching is obtained.

Position confirmation 3 In step 211, the reference image 3
01, a predetermined matching pattern P3 is cut out, and the moving image 306 is set at the same position as the matching pattern.
Is performed, and a result without matching is obtained.

In the matching determination step 212, it is determined whether or not there is a matching. In the position confirmation 2 step 210 and the position confirmation 3 step 211, it is determined that there is no matching. Position coordinates (Pm'nX, Pm'nY) of the displacement candidate area Pm'n of the displacement candidate area information having the next highest matching ratio obtained in the replacement step 206
And the position coordinates of the area of the matching pattern Pm (Pm
X, PmY) to calculate the displacement ΔPm, n (steps 208 to 213).

In the input image moving step 208, as shown in the displacement amount calculation image 305, the position information (Pm'nX, Pm'nY) of the next displacement candidate area Pm'n = P1'2 to be processed. ) = (P1′2X, P1′2)
Y) and the position coordinates (PmX, PmY) = (P1X, P1Y) of the area of the matching pattern Pm = P1 and the displacement amount ΔPm, n (Δm, nX, Δm, nY) = ΔP1,
2 (Δ1, 2X, Δ1, 2Y) is calculated from Expression (3) and Expression (4). Then, by moving the input image 302 by the calculated positional deviation amount ΔPm, n, the moving image 3
07 is obtained. Here, the moving image 307 is the input image 30
2 is obtained by moving Δ2 to the left by ΔP1,2.

In position confirmation 1 step 209, a preset matching pattern P1 is cut out from the memory image 301, and the moving image 30 is moved at the same position as the matching pattern.
7, and a result with matching is obtained.

Similarly, in the position confirmation 2 step 210 and the position confirmation 3 step 211, a result with matching is obtained.

In the matching determination step 212, it is determined whether or not there is a matching. Since the matching has been determined in the position checking 1 step 209, the position checking 2 step 210, and the position checking 3 step 211, the position shift amount calculation completion step 213 is performed. To perform the displacement candidate area P
m′n = Position shift amount ΔPm, n obtained from P1′2
(Δm, nX, Δm, nY) = ΔP1,2 (Δ1,2
X, Δ1, 2Y) are correct, and the process returns to the processing continuation determination step 207.

In the processing continuation determining step 207, it is determined that the calculation of the displacement amount is completed, and the
After determining the positional deviation amount ΔPm, n in step 4, the process returns to the input image capturing step 202. If the displacement amount ΔPm, n is within the predetermined value, the process returns to the input image capturing step 202.

Then, the next input image is fetched, and the detection of the positional shift of the camera by the above series of processing is continued.

[0044]

As described above, even when a pattern similar to a matching pattern exists in an input image, the position information of a plurality of matching patterns can be used.
Since the camera displacement can be calculated correctly and the camera displacement can be detected normally, it is possible to immediately notify that the camera is capturing an image outside the monitored area, and to detect undetected objects such as intruders and intruders. You can avoid it. In addition, it is possible to generate an image of one of the monitoring areas with high accuracy by sharing the images captured by sharing the monitoring area with a plurality of cameras, thereby facilitating the management of the image as monitoring information, Effective use can be achieved.

[Brief description of the drawings]

FIG. 1 is a flowchart of a camera position deviation detection processing according to an embodiment of the present invention.

FIG. 2 is an operation explanatory diagram of one embodiment of the present invention.

FIG. 3 is a block diagram of an image processing system according to an embodiment of the present invention.

FIG. 4 is a flowchart of a camera position shift detection processing according to the related art.

FIG. 5 is an explanatory diagram of the operation of the conventional technology.

FIG. 6 is a block diagram of a conventional image processing system.

[Explanation of symbols]

101: television camera, 102: image processing device,
103: image input I / F, 104: image memory, 10
5: reference image memory, 106: input image memory, 10
7: processing memory, 108: CPU, 109: output I /
F, 110: work memory, 111, 116: program memory, 112: image output I / F, 113: system bus, 114: alarm display monitor, 115: video monitor, 201: reference image capturing step, 202: input image capturing Step, 203: Positional deviation candidate position 1 calculation step, 204: Positional deviation candidate position 2 calculation step, 2
05: position shift candidate position 3 calculation step, 206: position shift candidate position rearranging step, 207: processing continuation determination step, 208: input image moving step, 209: position check 1 step, 210: position check 2 step, 21
1: Position confirmation 3 step, 212: Matching determination step, 213: Position deviation amount calculation completion step, 214:
Position shift amount determination step, 215: Alarm processing step, 301: Reference image, 302: Input image, 303: Search range, 304: Position shift candidate position image, 305: Position shift amount calculation image, 306: Moving image 1, 307 : Moving image 2, 401: displacement position calculation step, 402: displacement amount calculation step, P1, P2, P3: matching pattern, P1 ', P1'1, P1'2, P2'1, P
3'1: Position shift candidate area

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 7/18 H04N 7/18 K F term (Reference) 5C022 AA01 AB62 AC18 AC69 5C054 AA01 EA01 EA05 EA07 FA00 FC01 FC12 FC15 FE28 GB02 HA19 5L096 AA06 BA02 CA05 EA35 GA17 HA03 HA08 JA16 LA05

Claims (10)

[Claims] 1. A camera position shift amount calculating method for storing a plurality of input images captured by a camera in a storage medium and calculating a position shift amount of the camera from the images, wherein the plurality of input images are selected from the plurality of input images. A plurality of matching pattern areas are set from the reference image, and the position coordinates of the position error candidate area obtained by pattern matching between the input image and the matching pattern and the position coordinates of the matching pattern area are used for the camera. A method of calculating a camera position shift amount, comprising calculating a position shift. 2. The camera according to claim 1, wherein a plurality of positional deviation candidate areas having the highest matching rate in the pattern matching and their position coordinates are extracted, and the position coordinates of the camera are determined from the position coordinates of the matching pattern area. A method for calculating a camera position shift amount, comprising calculating a position shift amount. 3. The method according to claim 2, wherein the plurality of positional deviation candidate areas having the highest matching ratio and their position coordinates are rearranged in descending order of the matching ratio to calculate the positional deviation amount of the camera. A method of calculating the characteristic camera position shift amount. 4. A camera displacement detection method for storing a plurality of input images captured by a camera in a storage medium and referring to the images to detect the presence or absence of a displacement of the camera. The area of the plurality of matching patterns is set from the reference image selected from the above, and the position of the camera is determined from the position coordinates of the displacement candidate area obtained by pattern matching with the input image and the position coordinates of the area of the matching pattern. A method for detecting a camera position shift, comprising calculating a shift amount and detecting the presence or absence of a position shift of the camera based on a comparison between the position shift amount and a predetermined amount. 5. The position deviation amount of the camera according to claim 4, wherein a plurality of position deviation candidate areas having the highest matching ratio and position coordinates thereof are extracted, and the position deviation amount of the camera is obtained from position coordinates of the area of the matching pattern. Calculating the camera displacement and detecting the presence or absence of the camera displacement from the comparison between the displacement and a predetermined amount. 6. The method according to claim 5, wherein the plurality of positional deviation candidate areas having the highest matching ratio and their position coordinates are rearranged in descending order of the matching ratio to calculate the positional deviation amount of the camera. A method for detecting a camera position shift, comprising detecting the presence or absence of a position shift of the camera from a comparison between the position shift amount and a predetermined amount. 7. A moving image obtained by selecting a position shift amount of a camera having a higher matching ratio and moving the input image by the position shift amount. It is determined whether or not the plurality of matching patterns match at the same position as the area of the plurality of matching patterns, and if it is determined that no matching is performed, a position shift amount of the camera having the next highest matching ratio is selected. Then, it is determined whether or not matching is obtained by obtaining a moving image which is the same processing as described above, and if matching is determined, the presence / absence of the camera misalignment is detected by comparing the amount of misalignment with a predetermined amount. A method for detecting a camera position shift. 8. The method according to claim 7, wherein the amount of misalignment when it is determined that matching is performed is the amount of misalignment of the camera. 9. An image processing apparatus according to claim 1, wherein a plurality of input images picked up by the camera are taken in and image processing results are output to a means for displaying an image or notifying the result. A storage medium 1 in which a processing procedure of a calculation method is stored; and a CPU (C) that executes the processing procedure using the plurality of input images to generate a display image and camera position shift information.
an internal processing unit (hereinafter referred to as a CPU); a storage medium 2 for storing the plurality of input images and the display image; a conversion unit for converting the input image captured from the camera; Means for outputting information. 10. An image processing apparatus according to claim 4, wherein a plurality of input images picked up by a camera are taken in, and the result of image processing is output to a means for displaying an image or notifying the result. A storage medium 1 in which a processing procedure of a method is stored; and a CPU that executes the processing procedure using the plurality of input images to generate a display image and camera positional deviation information.
A storage medium 2 that stores the plurality of input images and the display image; a conversion unit that converts the input image captured by the camera into an input image; and a unit that outputs the display image and camera position shift information. An image processing apparatus characterized by the above-mentioned.