JP6022423B2 - Monitoring device and monitoring device control program - Google Patents

Description

  The present invention relates to a monitoring apparatus and a control program for the monitoring apparatus, and more particularly to an improvement of a monitoring apparatus that displays a camera image captured by a camera on a planar map.

  The monitoring device is a terminal device that displays a camera image captured by the monitoring camera on the monitor, and can monitor the imaging range of the monitoring camera. Some of such monitoring devices display the position and shooting range of the monitoring camera on a planar map, and control the direction and shooting magnification of the monitoring camera for which the user has specified the installation position on the planar map. There is also known a monitoring device capable of generating a mapping image by projective transformation of a camera image and displaying an image composite map obtained by superimposing the mapping image on a planar map (for example, Patent Document 1).

  Normally, when combining a camera image and a planar map, it is necessary to perform camera calibration that associates a position on the camera image with a position on the planar map. For example, camera calibration is performed by a user specifying a plurality of reference points on a camera image and a plurality of target points on a planar map, and associating these positional information between the camera image and the planar map. Thus, a conversion coefficient of the projective conversion is obtained.

  Such a conventional monitoring apparatus has a problem that the accuracy of camera calibration is low. For example, the camera image has lens distortion due to camera lens aberration. Further, the resolution of the camera image differs depending on the actual distance from the camera to the reference point on the horizontal plane, and the error included in the position information of the reference point increases as the distance from the camera increases. Furthermore, the digital calculation for obtaining the conversion coefficient from the position information includes a calculation error. Due to these effects, it has been difficult to accurately perform camera calibration.

JP 2005-286585 A

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a monitoring device capable of improving the accuracy of camera calibration between a camera image and a planar map. It is another object of the present invention to provide a monitoring apparatus control program that improves the accuracy of camera calibration between a camera image and a planar map.

  A monitoring device according to a first aspect of the present invention is a monitoring device that displays a camera image captured by a camera on a planar map, and performs projective transformation on the planar map to generate a mapping image. Projection transformation means, image synthesis means for generating an image synthesis map obtained by synthesizing the planar map and the mapping image, two or more reference points designated in advance on the camera image, and designation in advance on the planar map First calibration means for obtaining a conversion coefficient of the projective transformation for matching the reference point with the corresponding target point based on the two or more target points, and a composition for displaying the image composition map together with the target point Based on the map display means, the target point changing means for changing the position of the target point based on the user operation, and the target point after the change by the target point changing means, Constructed a second calibration means for correcting the 換係 number.

  In this monitoring device, by performing projective transformation using the transformation coefficient obtained by the first calibration, a mapping image in which an arbitrary position on the camera image is associated with a position on the planar map is obtained from the camera image. be able to. By displaying an image composite map in which such a mapping image and a plane map are combined, the user can determine whether or not there is a shift in the position of the feature portion between the mapping image and the plane map. The magnitude and direction of the deviation can be easily recognized. If the user changes the position of the target point on the image composite map as a result of the accuracy verification for the first calibration, the conversion coefficient of the projective transformation is corrected, so that the camera calibration between the camera image and the planar map is performed. Accuracy can be improved.

  In addition to the above-described configuration, the monitoring apparatus according to the second aspect of the present invention is configured such that the image synthesis unit generates an image synthesis map in which the transparent mapping image is superimposed on the planar map. According to such a configuration, the display object on the planar map can be recognized via the transparent mapping image, so that the degree of shift of the feature portion can be easily recognized between the mapping image and the planar map.

  In particular, the degree of displacement can be recognized over the entire mapping image, and it can be easily identified which part of the mapping image is greatly displaced. Therefore, when changing the position of the target point on the image composite map, it is possible to easily find a position where the deviation is reduced over the entire mapping image.

  The monitoring apparatus according to the third aspect of the present invention is configured to include a transparency adjusting means for adjusting the transparency of the mapping image in the image composite map based on a user operation in addition to the above configuration. According to such a configuration, the above-described accuracy verification of the first calibration can be facilitated by adjusting the transparency of the mapping image in accordance with the brightness and contrast of the camera image and the planar map, and image synthesis is performed. An appropriate position can be easily found when changing the position of the target point on the map.

  In addition to the above-described configuration, the monitoring device according to the fourth aspect of the present invention includes a distortion correction unit that performs distortion correction conversion for correcting lens distortion, and conversion of the distortion correction conversion based on a user operation. Correction coefficient adjusting means for adjusting the coefficient, and the projective conversion means is configured to perform the projective conversion on the camera image after the distortion correction conversion by the distortion correction means.

  According to such a configuration, the accuracy of camera calibration between the camera image and the planar map can be further improved by adjusting the conversion coefficient of the distortion correction conversion according to the degree of distortion of the camera image.

  A control program for a monitoring apparatus according to a fifth aspect of the present invention is a control program for a monitoring apparatus that displays a camera image captured by a camera on a planar map, and performs projective transformation of the camera image on the planar map. A projective transformation procedure for generating a mapping image, an image synthesis procedure for generating an image synthesis map obtained by synthesizing the planar map and the mapping image, two or more reference points designated in advance on the camera image, and the above A first calibration procedure for obtaining a conversion coefficient of the projective transformation for matching the reference point with the corresponding target point based on two or more target points specified in advance on a planar map, and the image together with the target point A composite map display procedure for displaying a composite map, a target point change procedure for changing the position of the target point based on a user operation, and a target point change procedure described above. Based on the target point after the change that, constituted by a second calibration procedure to correct the conversion factor.

  In the monitoring apparatus according to the present invention, if the user changes the target point on the image composite map, the conversion coefficient of the projective transformation is corrected, so that the accuracy of camera calibration between the camera image and the planar map can be improved. . In addition, the control program according to the present invention allows a computer to function as a monitoring device as described above.

1 is a system diagram showing a configuration example of a monitoring system 1 including a monitoring device 10 according to an embodiment of the present invention. It is the block diagram which showed an example of the function structure in the monitoring apparatus 10 of FIG. FIG. 3 is a diagram illustrating an example of the operation of the monitoring device 10 in FIG. 2, in which a camera image 13 and a map image 14 displayed on the display device 11 are illustrated. FIG. 3 is a diagram illustrating an example of the operation of the monitoring apparatus 10 in FIG. 2, and an image synthesis map 16 in which a map image 14 and a mapping image 15 are synthesized is illustrated. FIG. 3 is a diagram illustrating an example of the operation of the monitoring device 10 in FIG. 2, and shows a camera setting screen 30 displayed on the display device 11. FIG. 3 is a diagram showing an example of the operation of the monitoring apparatus 10 in FIG. 2, and shows an accuracy verification screen 40 displayed after the first calibration. It is the figure which showed an example of operation | movement of the monitoring apparatus 10 of FIG. 2, and the case where the transparency of the mapping image 15 is 50% is shown. FIG. 3 is a diagram showing an example of the operation of the monitoring apparatus 10 in FIG.

<Monitoring system 1>
FIG. 1 is a system diagram showing a configuration example of a monitoring system 1 including a monitoring device 10 according to an embodiment of the present invention. The monitoring system 1 includes two or more monitoring cameras 2 and a monitoring device 10 that connects these monitoring cameras 2 via a LAN (local area network) 20 and displays a camera image captured by the camera on a planar map. Consists of.

  The monitoring camera 2 is an imaging device that captures a subject and generates a camera image, and has a function of transmitting image data to the area monitoring device 10 via the LAN 20. For example, the camera image consists of a real-time moving image and is transmitted as a live video. The surveillance camera 2 can be installed either outdoors or indoors. In the case of outdoor installation, the surveillance camera 2 is installed toward a horizontal plane such as the ground or a road surface of a paved road, and a moving body on the horizontal plane is photographed. Mobile objects include people and cars. In the case of indoor installation, it is installed facing the horizontal surface such as the floor. For example, the surveillance camera 2 is a fixed camera with a fixed orientation and shooting magnification.

  The monitoring device 10 is a terminal device that includes a display device 11 that displays a camera image or a planar map acquired from the monitoring camera 2 and an operation unit 12 that receives a user operation, and performs area monitoring using the camera image. And has a function of controlling the surveillance camera 2 on the LAN 20. The operation unit 12 includes a mouse and a keyboard.

  The monitoring device 10 has a camera map display function, a detected object mapping function, and a plurality of video integration functions. The camera map display function is a function for displaying the position and shooting range of the monitoring camera 2 on a planar map. The detected object mapping function is a function that detects a moving object by analyzing a camera image, and displays the detected moving object and its trajectory on a planar map in real time.

  The multiple video integration function generates a mapping image by performing projective transformation that associates a position on the camera image with a position on the plane map for the camera image, and generates an image composite map obtained by combining the mapping image and the plane map. It is a function to display. For example, an image composite map in which a plurality of camera images acquired from two or more surveillance cameras 2 are embedded in a common plane map can be displayed. By browsing such an image composite map, it is possible to easily monitor or track a moving body that moves across the imaging ranges of the plurality of monitoring cameras 2.

  In order to realize the detected object mapping function and the multiple video integration function, it is necessary to perform camera calibration that associates position information with each other between the camera image and the planar map. In the monitoring device 10, the user designates two or more reference points on the camera image and two or more target points on the planar map, and associates the positional information between the camera image and the planar map to each other. Calibration is performed to obtain a conversion coefficient of the projective transformation.

<Monitoring device 10>
FIG. 2 is a block diagram illustrating an example of a functional configuration in the monitoring apparatus 10 of FIG. The monitoring device 10 includes a display device 11, a map information storage unit 101, a setting screen display unit 102, a projection conversion unit 103, an image composition unit 104, conversion coefficient storage units 105 and 113, a position designation unit 106, and a position information storage unit 107. , A first calibration unit 108, a target point changing unit 109, a second calibration unit 110, a transparency adjusting unit 111, a distortion correcting unit 112, and a correction coefficient adjusting unit 114.

  For example, the monitoring device 10 can be realized by operating a computer based on a control program. Further, such a control program is provided by being recorded on a computer-readable recording medium such as a CD-ROM or provided via a network.

  The map information storage unit 101 holds map information for displaying a planar map on the screen. For example, a map image made up of still images taken from above is used as a planar map. In addition to the map image, which consists of captured images, the planar map is composed only of figures such as building outlines, road boundaries, and paint lines indicating parking spaces, as well as figures and buildings and structures. A symbol composed of a symbol indicating an object or the like can be used.

  The setting screen display unit 102 displays a camera setting screen for camera calibration on the display device 11. The camera setting screen is a calibration input screen that displays the camera image acquired from the monitoring camera 2 and the map image stored in the map information storage unit 101 side by side, and receives the designation of the reference point and the target point. .

  The projective conversion unit 103 is a mapping image generation unit that performs projective conversion of a camera image on a planar map and generates a mapping image to be pasted on the planar map. This projective transformation is a geometric transformation in which two or more reference points on the camera image are associated with two or more target points on the planar map so that an arbitrary geometric shape on the horizontal plane in the camera image matches the planar map. .

For example, a coordinate axis composed of an x-axis and a y-axis is defined on the camera image, the position of the reference point is represented as (x, y), a coordinate axis composed of the X-axis and the Y-axis is defined on the plane map, and the target point If the position is expressed as (X, Y), the projective transformation of the camera image is expressed by the following equation (1).
X = (a ₁ x + b ₁ y + c ₁ ) / (a ₃ x + b ₃ y + c ₃ )
Y = (a ₂ x + b ₂ y + c ₂ ) / (a ₃ x + b ₃ y + c ₃ ) (1)

The above equation (1) includes nine parameters a _{1 to} a ₃ , b _{1 to} b ₃ , and c _{1 to} c ₃ representing the conversion coefficients of the projective transformation. In the above equation (1), even if the denominator is divided by a non-zero parameter among the parameters a _{3 to} c ₃ , for example, c ₃ , the relationship of the equation does not change. Therefore, if c ₃ = 1 is fixed, the remaining independent variables are eight. Accordingly, one projective transformation is determined by designating the eight parameters a _{1 to} a ₃ , b _{1 to} b ₃ , c ₁ and c ₂ . The conversion coefficient storage unit 105 holds parameters that define such projective transformation.

  The image composition unit 104 is a composite map generation unit that generates an image composite map obtained by combining the planar map stored in the map information storage unit 101 and the mapping image generated by the projective conversion unit 103. Specifically, an image composite map is generated by superimposing a mapping image on a planar map. The display device 11 displays the image composite map together with the target points.

  The position designating unit 106 designates two or more reference points on the camera image and two or more target points on the planar map based on a user operation, and the position information is assigned between the camera image and the planar map. To associate with each other. The position information storage unit 107 holds position information of reference points and target points and association information.

  The first calibration unit 108 performs projective transformation to match the reference point with the corresponding target point based on the reference point and the position information of the target point and the association information held in the position information storage unit 107. It is a camera image proofreading part which calculates | requires a coefficient. If four reference points where none of the three points are on the same straight line and four corresponding target points are designated, a conversion coefficient can be obtained.

Specifically, using Equation (1) for each combination of reference points and target points, eight equations are derived. That is, since two equations are established for the x and y coordinates with a set of reference points and target points, eight equations are obtained from a total of four sets of reference points and target points. Eight parameters a _{1 to} a ₃ , b _{1 to} b ₃ , c ₁ and c ₂ can be obtained by solving these equations simultaneously. By performing projective transformation using the transformation coefficient obtained by camera calibration, a mapping image in which an arbitrary position on the camera image is associated with a position on the planar map can be obtained from the camera image.

  Here, when camera images are acquired from two or more surveillance cameras 2 and these camera images are embedded on a common plane map to create an image composite map, a conversion coefficient for projective transformation is obtained for each surveillance camera 2. It is done. In addition, the image composition unit 104 generates an image composition map by superimposing a plurality of mapping images on a common plane map.

  The target point changing unit 109 is a position information editing unit that changes the position of the target point based on a user operation and updates the position information of the target point held in the position information storage unit 107. The target point changing unit 109 accepts a reference point change accompanying a change of the target point on the image composite map based on a user operation, and updates the position information of the reference point held in the position information storage unit 107. To do.

  The second calibration unit 110 corrects the transformation coefficient of the projective transformation based on the target point changed by the target point changing unit 109, and updates the parameters held in the transformation coefficient storage unit 105. Part. In addition, as a camera image used for the first calibration and the second calibration, a still image constituting the live video or a still image taken separately from the live video can be used instead of the live video.

  The transparency adjustment unit 111 adjusts the transparency of the mapping image in the image composite map based on a user operation. The image composition unit 104 generates an image composition map in which the transparent mapping image is superimposed on the planar map. The display device 11 displays an image composite map in which the mapping image is made transparent.

  The distortion correction unit 112 performs distortion correction conversion for correcting lens distortion on the camera image acquired from the monitoring camera 2, and outputs the camera image after distortion correction conversion to the setting screen display unit 102 and the projection conversion unit 103. It is a processing unit. Lens distortion is radial distortion (distortion aberration) of an optical lens.

For example, a coordinate axis composed of an x-axis and a y-axis is defined on the camera image, and the position on the camera image before and after the distortion correction conversion is defined as (x) with the position on the camera image corresponding to the position of the central axis of the optical lens as the origin. ₁ , y ₁ ), (x ₂ , y ₂ ), camera image distortion correction conversion is expressed by the following equation (2).
x ₂ = (1 + k ₁ r ² + k ₂ r ⁴ ) × x ₁
y ₂ = (1 + k ₁ r ² + k ₂ r ⁴ ) × y ₁ (2)
r ² = x ₁ ² + y ₁ ²

The above equation (2) includes two parameters k ₁ and k ₂ representing conversion coefficients for distortion correction conversion. By specifying these parameters k ₁ and k ₂ , distortion correction conversion is determined. However, the lens distortion, since the parameter k ₁ is practical to dominant, correction of lens distortion, by adjusting only the parameters k _1, can be performed in a simplified manner. Therefore, in the present embodiment, only the parameter k ₁ can be changed, and the parameter k ₂ is fixed or ignored, and the distortion correction conversion is performed by using the above equation (2). The conversion coefficient storage unit 113 holds parameters that define such distortion correction conversion. The correction coefficient adjustment unit 114 adjusts the conversion coefficient of the distortion correction conversion based on the user operation, and updates the parameter held in the conversion coefficient storage unit 113.

<Camera image 13 and map image 14>
FIG. 3 is a diagram illustrating an example of the operation of the monitoring device 10 in FIG. 2, and shows a camera image 13 and a map image 14 displayed on the display device 11. In the figure, (a) shows a camera image 13 acquired from the surveillance camera 2, and (b) shows a map image 14.

  The surveillance camera 2 is installed outdoors, for example, on the rooftop of a building, and images a moving body on the road surface. In this camera image 13, a paved road 13a, a building 13b, a car 13c stopped in the parking area, and the like are photographed. For the reference point 4 for the first calibration, a characteristic portion on the camera image 13 that can be easily associated with the map image 14 is designated.

  For example, the corner of the paint line indicating the parking area, the corner of the building 13b, the base of a structure such as a street light, etc. are designated as the reference point 4. The reference point 4 is designated by moving the mouse pointer 6 on the camera image 13 to indicate the position. Specifically, a rectangle having four reference points 4 as vertices is displayed as a default setting in the center of the camera image 13, and the reference point 4 is moved to a desired position by a drag operation.

  The map image 14 is a planar map that includes a part or all of the shooting range of the surveillance camera 2, and depicts a state in which the building 13b and the structure on the road surface are viewed from above in the vertical direction. In this map image 14, a figure representing the building 13 b, the traveling lane, the parking area, the shooting direction of the monitoring camera 2, and the symbol 3 indicating the installation position of the monitoring camera 2 are drawn.

  For the target point 5, a position on the map image 14 corresponding to the reference point 4 is designated. The target point 5 is designated by moving the mouse pointer 6 on the map image 14 to indicate the position. The first calibration is performed by obtaining transformation coefficients for projective transformation based on four combinations of the reference point 4 and the target point 5.

  On the camera image 13, the reference point 4 can be added by placing the mouse pointer 6 on a line segment connecting the two reference points 4 (indicated by a broken line in the figure) and performing a click operation. When the reference point 4 is added, the conversion coefficient of the projective transformation is obtained based on a combination of 5 or more including the reference point 4 and the target point 5. For example, it is repeated that four combinations are extracted from five or more combinations to obtain conversion coefficients, and an optimal value or average value of the conversion coefficients is calculated from the obtained conversion coefficients using a statistical method.

  Here, it has been described that the transformation coefficient of the projective transformation is obtained by designating four combinations of the reference point 4 and the target point 5 or five or more combinations. However, the number of reference points 4 and target points 5 specified by the user on the camera image 13 or the map image 14 is not limited to this. For example, when the height of the surveillance camera 2 from the horizontal plane (road surface) and the angle in the elevation angle direction are known, a reference mark or structure on the horizontal plane is photographed at a specific position on the camera image. In addition, when the position and orientation of the monitoring camera 2 are adjusted in advance, the number of reference points 4 and target points 5 designated by the user may be less than 4. In other words, if the shooting situation is limited, the conversion coefficient of the projective transformation may be determined by the user specifying two or three reference points 4 or target points 5.

<Image Composite Map 16>
FIG. 4 is a diagram showing an example of the operation of the monitoring apparatus 10 of FIG. 2, and shows an image synthesis map 16 in which the map image 14 and the mapping image 15 are synthesized. The mapping image 15 is created by performing projective transformation on the camera image 13. That is, the position information of an arbitrary pixel in the camera image 13 is converted into the position information of the pixel in the map image 14 by projective transformation.

  In this projective transformation, the geometric shape on the road surface is adjusted to the map image 14. For this reason, a stationary object in the vicinity of the road surface, such as a paint line of a paved road or a parking area, or a boundary line of the paved road, is converted into a shape when viewed from directly above and matches the shape in the map image 14.

  On the other hand, a stationary object or moving body protruding in the height direction from the road surface is transformed into a shape projected onto the road surface and appears distorted. The image composite map 16 is created as a composite image in which the mapping image 15 is superimposed on the map image 14 by arranging arbitrary pixels in the mapping image 15 at corresponding positions on the map image 14. By displaying such an image composite map 16 on the display device 11, the user can determine whether or not there is a shift in the position of the feature portion between the mapping image 15 and the map image 14, and if there is a shift, Can be easily recognized.

<Camera setting screen 30>
FIG. 5 is a diagram illustrating an example of the operation of the monitoring device 10 of FIG. 2, and shows a camera setting screen 30 displayed on the display device 11. The camera setting screen 30 is a calibration input screen for camera calibration, and includes a display area 31 for the camera image 13, a slider 32 for adjusting distortion correction, a display area 33 for the map image 14, the reference point 4, and A display button 34 for the target point 5 is arranged.

By operating the slider 32, the conversion coefficient of the distortion correction conversion for correcting the lens distortion can be adjusted for the camera image 13 in the display area 31. The conversion coefficient of the distortion correction conversion can be adjusted within a range from correction amount = zero to correction amount = upper limit value. When the correction amount = 0, the distortion correction conversion is not performed, and the camera image 13 passes through the distortion correction unit 112 and is input to the projection conversion unit 103. On the other hand, when the correction amount is equal to the upper limit value, the parameter k ₁ that defines the distortion correction conversion is maximized.

  The camera image 13 has lens distortion due to camera lens aberration. In particular, in the case of the surveillance camera 2 using a wide-angle lens, the distortion at the peripheral edge is larger than that at the center of the camera image 13. For this reason, the straight line on the horizontal plane is bent and projected at the peripheral edge of the camera image 13. Such distortion of the geometric shape photographed on the camera image 13 can be removed by operating the slider 32 and adjusting the correction amount.

  By operating the display button 34, the reference point 4 and the target point 5 for the first calibration can be displayed on the camera image 13 and the map image 14, respectively. The accuracy of the first calibration can be improved by specifying the reference point 4 after correcting the distortion of the camera image 13 and performing the first calibration.

<Accuracy verification screen 40>
FIG. 6 is a diagram illustrating an example of the operation of the monitoring apparatus 10 in FIG. 2, and shows an accuracy verification screen 40 displayed after the first calibration. FIG. 7 is a diagram showing an example of the operation of the monitoring apparatus 10 of FIG. 2, and shows a case where the transparency of the mapping image 15 is 50%.

  This accuracy verification screen 40 is a camera calibration editing screen for verifying the accuracy of the correspondence between the position information between the camera image 13 and the map image 14. The display area 41 of the camera image 13 and the image composite map 16 A display area 42, a slider 43 for adjusting the transparency of the mapping image 15, and a correction calibration button 44 are arranged.

  By operating the slider 43, the transparency of the mapping image 15 can be adjusted with respect to the image composite map 16 in the display area 42. The transparency of the mapping image 15 can be adjusted within a range from transparency = 100% to transparency = 0%. When the transparency is 100%, the mapping image 15 is completely transparent, and the entire map image 14 can be visually recognized. On the other hand, when the transparency is 0%, the mapping image 15 is completely opaque, and the map image 14 cannot be visually recognized with respect to the overlapping portion with the mapping image 15.

  If the transparency is about 50%, the mapping image 15 is translucent. The display object on the map image 14 can be recognized through the transparent mapping image 15 by displaying the image composite map 16 in which the transparent mapping image 15 is superimposed on the map image 14 in this way. Thus, it is possible to easily recognize the deviation of the feature portion between the mapping image 15 and the map image 14.

  In the display area 41, a plurality of reference points 4 used in the first calibration are displayed together with the camera image 13. In the display area 42, a plurality of target points 5 used in the first calibration are displayed together with the image composite map 16. Therefore, if there is a shift in the position of the feature portion between the map image 14 and the mapping image 15, the mouse pointer 6 is moved, and the position of the reference point 4 on the camera image 13 and the target on the image composite map 16 are moved. The position of the point 5 can be changed.

  FIG. 8 is a diagram illustrating an example of the operation of the monitoring apparatus 10 in FIG. 2, and illustrates a case where the position of the characteristic portion is shifted between the map image 14 and the mapping image 15. Regarding the horizontal plane at a predetermined distance from the monitoring camera 2, the geometric shape in the horizontal plane on the camera image 13 is different from the actual geometric shape due to lens distortion of the camera image 13. Further, since the resolution of the camera image 13 varies depending on the actual distance from the monitoring camera 2 to the reference point 4 on the horizontal plane, the position information of the reference point 4 far from the monitoring camera 2 includes a large error. Further, the digital calculation when obtaining the conversion coefficient of the projective transformation from the position information includes a calculation error.

  For this reason, in the image composition map 16, there may be a deviation between the position of the feature portion on the mapping image 15 and the position of the corresponding feature portion on the map image 14. In this example, the position of the corner (corner) of the building 13 b is greatly shifted between the mapping image 15 and the map image 14. Such a shift can be easily identified by browsing the image composite map 16 in which the transparent mapping image 15 is embedded.

  The first calibration was performed by designating the reference point 4 and the target point 5 using the paint line indicating the parking area as a mark. In the first calibration, it can be easily confirmed from the image composite map 16 that there is little deviation in the vicinity of the parking section, but that the deviation is large at the corner of the building 13b photographed at the periphery of the camera image 13. The large deviation of the corner of the building 13b is due to the influence of lens distortion and calculation error.

  Therefore, the second calibration is performed by finely adjusting the position of the target point 5 in the parking area on the image composite map 16 or finely adjusting the position of the reference point 4 in the parking area on the camera image 13. The second calibration is executed by operating the correction calibration button 44 in the accuracy verification screen 40, and the conversion coefficient of the projective transformation is corrected. After this correction calibration, the angle deviation of the building 13b can also be easily confirmed by the image composite map 16. By repeating such correction calibration, the positional deviation is reduced at points other than the four points used in the first calibration, and the positional deviation from the map image 14 can be reduced over the entire mapping image 15. .

  Further, if necessary, the target point 5 is additionally specified near the corner of the building 13b, the corresponding reference point 4 is also specified on the camera image 13, and the correction calibration is performed by the five sets of the reference point 4 and the target point 5. Can also be performed. If the reference point 4 or the target point 5 is changed, the correction calibration may be automatically executed.

  According to the present embodiment, by performing projective transformation using the transformation coefficient obtained by the first calibration, the mapping image 15 in which an arbitrary position on the camera image 13 is associated with a position on the planar map. Can be obtained from the camera image 13. By displaying the image composite map 16 in which such a mapping image 15 and the planar map are combined, the accuracy verification of the first calibration can be facilitated. Further, if the user changes the position of the target point 5 on the image composite map 16 as a result of the accuracy verification for the first calibration, the conversion coefficient of the projective transformation is corrected, so that the camera between the camera image 13 and the planar map is corrected. Calibration accuracy can be improved.

  Furthermore, since the display object on the planar map can be recognized via the transparent mapping image 15, it is possible to easily recognize the deviation of the feature portion between the mapping image 15 and the planar map. Further, by adjusting the transparency of the mapping image 15 according to the brightness and contrast of the camera image 13 and the planar map, the accuracy verification of the first calibration can be facilitated, and the target point on the image composite map 16 can be obtained. When changing the position of 5, an appropriate position can be easily found.

  In the present embodiment, an example in which the image composite map 16 is created by superimposing the mapping image 15 on the plane map has been described. However, the present invention is an image composite map obtained by combining the plane map and the mapping image. The configuration is not limited to this. For example, when the planar map is composed of figures or symbols, or when a transparent planar map is used, the configuration may be such that the image composite map is formed by superimposing the planar map on the mapping image. .

  In the present embodiment, an example in which the user specifies the positions of the reference point 4 and the target point 5 used for the first calibration has been described. However, the present invention describes a method for specifying the reference point 4 and the target point 5. However, the present invention is not limited to this. For example, the configuration may be such that a feature portion is automatically extracted from the camera image 13 and the map image 14 and the reference point 4 and the target point 5 are designated based on the extracted feature portion.

  The feature portion on the image includes an edge and a corner where two edges intersect, and is extracted based on the luminance distribution and the luminance change rate. Further, by displaying the edges and corners extracted from the camera image 13 and the map image 14 on the image composite map 16, it is possible to easily recognize the deviation of the feature portion between the mapping image 15 and the map image 14.

  In addition, if there is a deviation between the edge or corner position extracted from the camera image 13 and the edge or corner position extracted from the map image 14, the user recognizes the degree of match between the camera image 13 and the map image 14. In order to achieve this, a display mode of symbols representing edges and corners, for example, a configuration in which the color is varied depending on the magnitude of the shift may be used.

  In the present embodiment, an example in which the monitoring device 10 includes a terminal device including the display device 11 and the operation unit 12 has been described. However, the present invention does not limit the configuration of the monitoring device 10. . For example, a computing device that acquires and processes camera images from the monitoring camera 2 and a terminal device that displays camera images and processing results and receives user operations are connected to a monitoring system connected via a communication network such as the Internet. The present invention can also be applied.

DESCRIPTION OF SYMBOLS 1 Monitoring system 2 Monitoring camera 4 Reference point 5 Target point 6 Mouse pointer 10 Monitoring apparatus 11 Display apparatus 12 Operation part 13 Camera image 14 Map image 15 Mapping image 16 Image composite map 101 Map information storage part 102 Setting screen display part 103 Projection conversion Unit 104 image composition unit 105, 113 conversion coefficient storage unit 106 position designation unit 107 position information storage unit 108 first calibration unit 109 target point change unit 110 second calibration unit 111 transparency adjustment unit 112 distortion correction unit 114 correction coefficient adjustment Part 20 LAN

Claims (5)

In a monitoring device that displays a camera image captured by a camera on a planar map,
Projective transformation means for projectively transforming the camera image with respect to the planar map to generate a mapping image;
Image synthesis means for generating an image synthesis map obtained by synthesizing the planar map and the mapping image;
The projective transformation for matching the reference point with the corresponding target point based on the two or more reference points specified in advance on the camera image and the two or more target points specified in advance on the planar map. First calibration means for obtaining a conversion coefficient of
Synthetic map display means for displaying the image composite map together with the target point;
Target point changing means for changing the position of the target point based on a user operation;
A monitoring apparatus comprising: second calibration means for correcting the conversion coefficient based on the target point changed by the target point changing means.   The monitoring apparatus according to claim 1, wherein the image composition unit generates an image composition map obtained by superimposing the transparent mapping image on the planar map.   The monitoring apparatus according to claim 2, further comprising: a transparency adjusting unit that adjusts transparency of the mapping image in the image composite map based on a user operation. Distortion correction means for performing distortion correction conversion for correcting lens distortion for the camera image;
Correction coefficient adjusting means for adjusting the conversion coefficient of the distortion correction conversion based on a user operation,
The monitoring apparatus according to claim 1, wherein the projective conversion unit performs the projective conversion on the camera image after the distortion correction conversion by the distortion correction unit. A control program for a monitoring device for displaying a camera image taken by a camera on a planar map,
Projection conversion procedure for projectively converting the camera image to generate a mapping image for the planar map,
An image synthesis procedure for generating an image synthesis map obtained by synthesizing the planar map and the mapping image;
The projective transformation for matching the reference point with the corresponding target point based on the two or more reference points specified in advance on the camera image and the two or more target points specified in advance on the planar map. A first calibration procedure for obtaining a conversion coefficient of
A composite map display procedure for displaying the image composite map together with the target point;
A target point changing procedure for changing the position of the target point based on a user operation;
A control program for a monitoring apparatus, which causes a computer to execute a second calibration procedure for correcting the conversion coefficient based on the target point after the change in the target point changing procedure.

Images