JP3288575B2 - Monitoring apparatus using captured image, model creation method thereof, and imaging apparatus control method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus for capturing an image of a site such as a factory or a plant using an image pickup device, outputting the image to a monitor operator's display device, and using the picked-up image as an input screen for monitoring operation. The present invention relates to a monitoring device using a captured image, a model creation method thereof, and an imaging device control method.

[0002]

2. Description of the Related Art FIG.
FIG. 6 is a configuration diagram showing a conventional monitoring apparatus using a captured image disclosed in Japanese Patent Application Publication No. 6 (JP-A-6), in which 1 is a computer. Reference numeral 2 denotes an imaging device, 3 denotes a display device, 4 denotes a pointing device, and 5 denotes a stereophonic system, which are connected to the computer 1. Reference numeral 6 denotes a microphone, 7 denotes headphones, and these are connected to the stereophonic sound system 5.

[0003] The conventional method of detecting an object by the monitoring apparatus using the captured image configured as described above uses a computer 1 that calculates a model of the shape and arrangement of the monitored object at the site of the imaging object to be imaged.
Created above, when one point in the display device 3 is specified, the point is made to correspond to a straight line in the model from the parameters of the imaging device 2, and an object in the model that intersects the straight line is detected. is there.

Next, the operation will be described. The monitoring device using the captured image displays the captured image captured by the imaging device 2 whose imaging parameter can be controlled by the computer 1 on the display device 3, and the monitoring target device is a pointing device 4 such as a mouse. FIG. 24 is a flowchart showing the procedure of the processing.

First, when the process is started in step ST1, in step ST2, the position of the device to be monitored is input by the pointing device 4 or the like on the screen of the display device 3 on which the captured image is displayed. I do. Next, in step ST3, the input position is detected. In step ST4, a line of sight in a three-dimensional space in which a plane model that models the imaging target is obtained from the position data. In step ST5, the intersection of the line of sight and the plane is obtained. Ask for.
In this manner, an intersection is obtained between a plurality of planes. Next, a plane closest to the viewpoint among the plurality of intersections is detected in step ST6. Then, in step ST7, it is detected which of the monitoring target models of the monitoring target models drawn in the plane model has the intersection of the plane.

[0006] In addition to the above, there is another conventional monitoring apparatus using a captured image as disclosed in Japanese Patent Application Laid-Open No. 7-264458. In this monitoring device, detection of an object is performed as follows. That is, a model of the shape of the monitoring target at the site of the imaging target to be imaged is created on a computer, the captured image is image-processed, and the result of the image processing and the model of each monitoring target are monitored from the image. The object is recognized and detected.

Further, as a control method of an imaging device used in such a monitoring device using a captured image, for example, a method described in JP-A-7-274150 is known.
Alternatively, there is a method described in JP-A-7-319536. The former method, that is, JP-A-7-274
The control of the imaging device described in Japanese Patent Publication No. 150 is performed such that the user creates a region of interest on the display screen of the display device 3 on which the captured image is displayed, and the region is displayed on the entire screen. The image pickup device 2 is automatically controlled. In addition, the latter method, that is,
The control of the imaging device described in JP-A-319536 is
On the display screen of the display device 3 on which the captured image is displayed,
By instructing a monitoring device to be focused on, zoom-in, zoom-out, and rotation of the imaging device 2 are automatically controlled depending on where a certain representative point of the monitoring device is located on the display screen.

[0008]

Since the conventional monitoring apparatus using a captured image is configured as described above, there are the following problems. That is, JP-A-7-31953
In the detection of the monitoring target object disclosed in Japanese Patent Application Laid-Open No. 6-64, the portion to be selected is a point on the display screen of the display device 3 and the monitoring target object cannot be detected by the selected position due to an error between the display of the captured image and the model. In addition, there is a problem that work such as occurrence of an image or enlargement of a captured image is required, and it is not possible to cope with a case where the monitored object moves. In addition, in the detection of a monitoring target disclosed in Japanese Patent Application Laid-Open No. 7-264458, the recognition rate of image recognition is affected by the environment of the imaging target, such as the illumination of the site, the color of the target, and dirt. There was a problem that it might not be possible.

Further, in the control of the image pickup apparatus disclosed in Japanese Patent Application Laid-Open No. 7-274150, the entire monitor target is not displayed as a picked-up image on the display device 3 because it does not have an image pickup target model. In the control of the imaging device described in JP-A-7-319536, only the representative point of the monitoring target in the model is used for controlling the imaging device 2. Since the size of the monitoring target is not taken into consideration, there is a problem that the imaging device 2 that is suitable for the monitoring target cannot be controlled.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems, and has a model of an imaging target in order to make a correspondence between a monitoring target site and a captured image.
It is an object of the present invention to obtain a monitoring device using a captured image that can detect a monitoring target even when there is an error between the captured image and the model by enabling the user to specify an imaging target in an area set by a user. I do.

It is another object of the present invention to provide a monitoring apparatus using a captured image which can cope with a moving object by reflecting the actual movement in a model even when the monitored object moves. I do.

It is a further object of the present invention to provide a model creation method capable of creating an imaging target model by directly using a captured image of a monitored object.

A further object of the present invention is to provide a method of controlling an image pickup apparatus that automatically controls the image pickup apparatus in accordance with the position and size of an object to be monitored.

[0014]

According to a first aspect of the present invention, there is provided a monitoring apparatus using a captured image, which includes an imaging target model obtained by modeling an actual site, and a display partial area in a display screen of a display device. From the display partial region instruction operation unit, the imaging partial region calculation unit obtains the imaging partial region of the imaging device from the display partial region, and the model partial region calculation unit calculates from the obtained imaging partial region and the imaging target model. The model to be monitored in the model partial area is extracted by the model extraction unit, the monitoring information output processing unit recognizes the actual monitoring object at the actual site from the monitoring target model, and displays the information of the monitoring object. This is displayed on a display device via a control processing unit.

According to a second aspect of the present invention, a monitoring apparatus using a captured image uses an input imaging parameter and an imaging partial area of the imaging apparatus and causes a model partial area calculation unit to display an imaging projection plane of an imaging target model in a model partial area calculation unit. A model partial area is created by arranging the model partial area, and a model extraction unit extracts a monitoring target model overlapping the model partial area as a monitoring target.

According to a third aspect of the present invention, there is provided a monitoring apparatus using a captured image, wherein a three-dimensional model is used as an imaging target model, and a three-dimensional model is used as a model partial area in a model partial area calculation unit.
A 3D model is created, and the model extractor
The monitoring target model is extracted based on the interference between the three-dimensional model of the monitoring target and the three-dimensional model of the monitoring target.

According to a fourth aspect of the present invention, when there are two or more monitored models that interfere with the three-dimensional partial model, they are sorted and displayed according to a predetermined rule. The function to be displayed on the device is provided to the model extraction unit.

According to a fifth aspect of the present invention, there is provided a monitoring apparatus using a captured image, wherein the monitoring information output processing unit has a function of changing a three-dimensional model as an imaging target model based on an actual site condition. Thus, the three-dimensional model is changed so as to simulate the actual movement of the monitored object.

According to a sixth aspect of the present invention, there is provided a monitoring apparatus using a picked-up image, wherein designation information for designating a measurement portion is inputted from a display partial region designation operation section, and the designation information is modeled by a model partial region calculation section. It is converted into a partial area to perform measurement calculation in a three-dimensional model.

According to a seventh aspect of the present invention, a monitoring apparatus using a captured image uses a surface model as an imaging target model.
The model partial area calculation unit uses the imaging partial area calculated by the imaging partial area calculation unit to create a model partial area corresponding to the imaging partial area on the surface model, and the model extraction unit A monitoring target model that interferes is obtained and used as a monitoring target object to be extracted.

According to an eighth aspect of the invention, there is provided a model creating method for converting a contour of a monitoring target object obtained by directly indicating a monitoring target object projected on a display screen into a contour on an imaging projection plane. Then, the monitoring target model is created by converting it into a contour figure of the monitoring target model.

According to a ninth aspect of the present invention, there is provided a model creating method, further comprising calculating a position of each monitored object model on a display device, and displaying graphical information indicating that the monitored object model has been created. This is displayed so as to be superimposed on the captured image at a position on the display device corresponding to the position of the monitoring target model.

According to a tenth aspect of the present invention, a monitoring apparatus using a captured image obtains a point of interest of a monitoring target model extracted from the imaging target model in a monitoring information output processing unit, and determines the point of interest as a captured image. This is converted into a point on the displayed display screen, and the monitoring information of the monitoring target corresponding to the position is displayed so as to overlap the captured image.

According to an eleventh aspect of the present invention, there is provided a method for controlling an imaging apparatus, wherein a point of interest of a monitoring target model extracted from an imaging target model is obtained, and is converted into a direction vector in a coordinate system of the imaging apparatus. A drive control parameter for directing the imaging device in the direction of the direction vector is calculated and provided to the drive control device of the imaging device to control the rotation of the imaging device.

According to a twelfth aspect of the invention, there is provided a method for controlling an imaging apparatus, wherein a monitoring target model in an imaging target model is extracted based on the input identifier of the monitoring target.

According to a thirteenth aspect of the present invention, a monitoring target model is extracted from the imaging target model based on the position on the display screen of the monitoring target specified by the monitoring operator. It is like that.

According to a fourteenth aspect of the present invention, there is provided an imaging apparatus control method for extracting a monitoring target model from an imaging target model based on a detection result of whether or not an abnormality or a failure has occurred in an actual monitoring target at a site. It is something to do.

According to a fifteenth aspect of the present invention, a minimum two-dimensional figure surrounding the projected figure of the monitoring target model extracted from the imaging target model is obtained.
A drive control parameter for zooming in or out of the imaging device is calculated so that the three-dimensional figure matches the imaging region of the imaging device, and the calculated drive control parameter is provided to the drive control device of the imaging device to control the enlargement or reduction of the imaging device. It is something to do.

[0029]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below. Embodiment 1 FIG. FIG. 1 is a block diagram showing a configuration of a monitoring device using captured images according to Embodiment 1 of the present invention. In the figure, reference numeral 2 denotes an imaging device for imaging an actual site such as a factory or a plant to be monitored, and reference numeral 3 denotes a display device for displaying a captured image of the monitoring target captured by the imaging device 2. Note that the imaging device 2 and the display device 3 are portions corresponding to those in the related art, which are denoted by the same reference numerals in FIG.

Reference numeral 10 denotes a display partial region instruction operation unit for designating a display partial region on the display screen of the display device 3. The pointing device indicated by reference numeral 4 in FIG. Used. 11 is the display device 3
Is a display control processing unit that processes display control and display state management for the display device, displays a captured image at a desired position on the display device 3, displays a cursor of a pointing device such as a mouse or a tablet, and displays a display portion. It performs processing such as displaying a graphic of the display partial area specified by the area specifying operation unit 10 and further performing display control for displaying monitoring information. Reference numeral 12 denotes an imaging partial area calculation unit that obtains an imaging partial area of the imaging device 2 from the display partial area specified by the display partial area instruction operation unit 10, and captures an image of the display partial area created in the coordinate system of the display apparatus 3. An imaging partial area on the projection plane is calculated.

Reference numeral 13 denotes an imaging target model that models the real world as the imaging target. Reference numeral 14 denotes a model partial area calculation unit that calculates a model partial area that is a partial area in the imaging target model 13;
Imaging partial area and imaging target model 13 calculated in
Calculates the model partial region from. Reference numeral 15 denotes a model extraction unit that extracts a monitoring target model in the model partial area calculated by the model partial area calculation unit 14. The monitoring target object is located within the model partial area from the imaging target model 13 and the model partial area. Extract the model of Reference numeral 16 denotes a monitoring information output processing unit that outputs information of a monitoring target according to an instruction of a monitoring worker. The monitoring information output processing unit 16 recognizes an actual monitoring target at the site from the monitoring target model extracted by the model extracting unit 15, and performs monitoring of the monitoring target. Is output to the display device 3 via the display control processing unit 11.

FIG. 2 is an explanatory diagram for explaining the processing in the monitoring apparatus using the captured image according to the first embodiment configured as described above. In the figure, 21
Denotes a display screen of the display device 3, and 22 denotes a captured image displayed on the display screen 21. Reference numeral 23 denotes a display partial area created in the display screen 21 by the monitor operator using the display partial area instruction operation unit 10, and 24 denotes a monitoring target indicated by the display partial area 23. Reference numeral 25 denotes an imaging projection plane. Reference numeral 26 denotes an imaging partial area created on the imaging projection plane 25 by the imaging partial area calculation unit 12 based on the display partial area 23. Reference numeral 27 denotes an imaging target model indicated by reference numeral 13 in FIG. 1, and reference numeral 28 denotes a model partial area calculation unit 14 which is arranged by using an imaging position and an imaging direction of the imaging apparatus 2 and imaging parameters such as enlargement and reduction. The imaging projection plane in the imaging target model 27 is a model partial area in the imaging target model 27. 30 is a model extraction unit 1
5 is a monitoring target model included in this model partial area extracted by the model sub-region 5.

Next, the operation will be described. FIG. 3 is a flowchart showing a procedure of the process in the first embodiment. When the process is started in step ST10, a captured image 22 captured by the imaging device 2 is displayed on the display screen 21 of the display device 3. In step ST11, the monitoring operator uses the display partial area instruction operation unit 10 to display a display partial area 23 on the display screen 21 of the display device 3 on which the captured image 22 is displayed.
Instruct. Next, the imaging partial area calculation unit 12 performs step S
At T12, the display management data of the display control processing unit 11 and the display screen 2
1. From the display partial area 23 indicated on the
5 is calculated. The imaging partial area 26 calculated by the imaging partial area calculation unit 12 is passed to the model partial area calculation unit 14 together with the imaging position, the imaging direction, and imaging parameters such as enlargement and reduction, and the model partial area calculation unit 14 proceeds to step ST13. , An imaging target model 27 using the imaging partial area 26 and each imaging parameter.
A model partial area 29 is calculated by arranging the imaging projection plane 28 in the inside.

Next, the model extracting section 15 executes step ST14.
, The overlap between the model partial region 29 calculated by the model partial region calculation unit 14 and the model of each monitoring target in the imaging target model 13 is compared.
9 and the monitoring target model 30 included therein are extracted. Next, the process proceeds to step ST15, where the monitoring target model 30 extracted by the model extracting unit 15 is
The monitoring information output processing unit 16 recognizes the monitoring target 24 at the actual site corresponding to the monitoring target model 30 based on the monitoring information, and sends the monitoring information of the monitoring target 24 to the display control processing unit 11. The monitoring information about the monitoring target model 30 is displayed on the display device 3. In this way, a series of monitoring operations are performed, and the process ends in step ST16.

As described above, according to the first embodiment, a partial area of a monitored object is specified and extracted from a display screen on which a captured image is displayed.
Even if there is an error between the captured image and the imaging target model corresponding to the captured image, it is possible to reliably extract the monitoring target object, and it is not necessary to select a monitoring target object. Even when a plurality of areas are specified, there is an effect that the specification can be made by specifying one partial area.

Embodiment 2 In the second embodiment, a three-dimensional model created in a computer is used as the imaging target model 13. Since the device configuration is the same as that of the first embodiment shown in FIG. Description is omitted.

FIG. 4 is a schematic diagram showing an example of an imaging target model used in the second embodiment of the present invention. In the figure, reference numeral 40 denotes a model space of a three-dimensional model which is the imaging target model 13, and reference numeral 41 denotes an imaging projection plane calculated using imaging parameters such as an imaging position, an imaging direction, and enlargement and reduction with respect to the model space 40. , 42 are the imaging projection surface 41
An imaging apparatus having a certain imaging direction.

FIG. 5 is a schematic diagram showing a model partial area according to the second embodiment. In the figure, reference numeral 43 denotes an imaging partial area in the imaging projection plane 41 when the imaging projection plane 41 is arranged in a three-dimensional model space 40 that is the imaging target model 13, and 44 denotes an imaging position, an imaging direction, and an enlargement. This imaging partial area 43 is obtained by using imaging parameters such as
This is a three-dimensional model of the photographing region created as a model partial region.

Next, the operation will be described. FIG. 6 is a flowchart showing a procedure of a process according to the second embodiment. First, in step ST20, the imaging projection plane 41 and the viewpoint of the imaging device 42 arranged at the same position and orientation as the actual site are set using the imaging parameters in the model space 40 of the three-dimensional model. Then step ST
At 21, the model partial region calculation unit 14 determines, based on the imaging partial region 43 calculated by the imaging partial region calculation unit 12 in the imaging projection plane 41 and the viewpoint of the imaging device 42, the 3 A dimensional model 44 is created.

Then, the model extracting unit 15 executes step ST2.
In 2, an interference check between the three-dimensional model 44 as the model partial region and the monitoring target model of the monitoring target in the imaging target model 13 is performed. In this interference check, for example, an interference calculation realized by many three-dimensional CADs (computer-aided design) is performed. Next, in step ST23, it is determined whether or not the monitoring target model that interferes is detected by the interference check. As a result, if there is no interfering monitoring target model, the fact is notified to the monitoring operator in step ST24, and the process is terminated.

On the other hand, if there are two or more monitoring target models that interfere with each other, the process proceeds to step ST25, where sorting is performed according to the rule intended for the operation, for example, from the one near the viewpoint or from the one with a high degree of interference. Then, in step ST26, the contents of the sorting are sent to the monitoring information output processing unit 16, and displayed on the display device 3 via the display control processing unit 11 to notify the monitoring worker. In step ST27, the monitoring operator checks the display device 3
Select an appropriate monitoring target from the monitoring target models displayed in. If only one interfering monitoring target model is detected, one of them is automatically selected.

As described above, according to the second embodiment, since a three-dimensional model of the actual site space and the monitored object is provided as the captured object model, the captured image is hidden behind a certain monitored object. Can be specified, the range of the monitoring target can be expanded, and even if the arrangement position of the imaging device is changed by creating one three-dimensional model, Because it is possible to rebuild the system by changing the imaging parameters,
It is possible to flexibly respond to changes in the system, and it is possible to easily add an imaging device. Further, when there are two or more models of the monitoring target that interfere with each other, the monitoring worker sorts the display according to the rule according to the intention of the operation and displays it on the display device, so that the monitoring operator can model the monitoring target. It becomes possible to specify and select an appropriate one from among them.

Embodiment 3 In the third embodiment, the three-dimensional model created in the computer as the imaging target model 13 is changed in accordance with the actual on-site state.
The dimensional model simulates the movement of the monitoring target. FIG. 7 is a block diagram showing a configuration of a monitoring apparatus using a captured image according to Embodiment 3 of the present invention, where the same reference numerals are assigned to the corresponding parts as in Embodiment 1 shown in FIG. Description is omitted. The monitoring information output processing unit 16 has a function of changing the three-dimensional model as the imaging target model 13 so as to simulate the movement of the actual monitoring target based on the read actual site condition. This is different from the same reference numerals in FIG.

FIG. 8 is a flowchart showing the procedure of the process of the monitoring information output processing unit 16 according to the third embodiment. In step ST30, the monitoring information output processing unit 16 first reads the states of the sensors and devices from the actual site and acquires information related to the operation of the monitoring target. Next, in step ST31, an arrangement parameter in the three-dimensional model as the imaging target model 13 is obtained from the obtained information related to the operation of the monitoring target. Next, in step ST32, the arrangement of the monitoring target model in the three-dimensional model created as the imaging target model 13 is updated according to the obtained arrangement parameters so as to simulate the actual movement of the monitoring target.

As described above, according to the third embodiment, the arrangement of the monitoring target model is changed in accordance with the moving monitoring target, so that a single imaging parameter such as a transfer device or a robot can be used. Even if there is a monitoring target whose imaging image changes dynamically, the imaging target model can be dynamically changed in the same manner as the monitoring target that dynamically changes, so that the monitoring target that dynamically changes An object can be selected, and there is an effect that a monitoring operation that does not limit the type of the monitoring target can be performed.

Embodiment 4 FIG. In the fourth embodiment, the display partial area instruction operation unit 10 has a function of inputting designation information for designating a measurement unit such as a length and a region for a captured image projected on the display device 3, and the designation information is provided. Acquisition of measurement values such as length, area, volume, etc. for the actual site by providing the function of converting into a model partial area and performing measurement calculation in the three-dimensional model in the model partial area calculation unit 14 Is made possible. Note that the configuration of the apparatus is basically the same as that of the first embodiment shown in FIG. 1, and a description thereof will be omitted here.

FIG. 9 is an explanatory diagram showing an example of measurement processing according to the fourth embodiment of the present invention. Here, the case where the three-dimensional model shown in the second embodiment is used is shown. 9A shows an example of processing when measuring the distance between two specified points, FIG. 9B shows an example of processing when measuring the area of a specified area, and FIG. FIG. 9D shows a processing example when measuring the height of the designated object, and FIG. 9D shows a processing example when measuring the volume of the designated portion.

That is, in the case of FIG. 9A, it is desired to measure the distance between the two by the display partial area instruction operation unit 10.
Two points DP1 and DP2 are input on the display screen. From the points DP1 and DP2, the method described in the second embodiment is used.
The straight lines ML1 and ML2 in the three-dimensional model are created. Then, the surveillance worker is caused to select a plane at which the straight lines ML1 and ML2 intersect, and the intersections MP1 and MP2 between the selected plane and the straight lines ML1 and ML2 are calculated. This calculated intersection M
By calculating the distance between P1 and MP2, it is used as a measured value of the distance between two points DP1 and DP2 to be measured.

In the case of FIG. 9B, the contour DS of the area of the surface to be measured is input on the display screen by the display partial area instruction operation unit 10. Embodiment 2 from the outline DS
A three-dimensional model MD in the three-dimensional model is created by the method shown in (1), and the surface overlapping with the three-dimensional model MD is shown to the monitoring operator to be selected. Then, an area MF where the selected plane intersects with the three-dimensional model MD is obtained, and the area of the area MF is calculated to obtain a measured value of the area of the area to be measured.

Further, in the case of FIG. 9 (c), a point DP1 at a position to be measured by the display partial area instruction operation unit 10
Area DS for selecting a reference plane for referencing height
Is input on the display screen. Then, the point MP in the three-dimensional model is obtained in the same manner as in FIGS. 9A and 9B.
1 and the surface MS are obtained. Then, a straight line ML is created in the vertical direction from the point MP1, and an intersection M between the straight line ML and the surface MS is created.
Find P2. The height is calculated from the distance between the point MP1 and the intersection MP2, and the calculated value is used as a measured value of the height of the position to be measured.

In the case of FIG. 9D, FIG.
As in the case of (a), two points DP1 and DP2 are input on the display screen, and two points MP1 and M2 in the three-dimensional model are input.
Find P2. Then, a rectangular parallelepiped MB having these points MP1 and MP2 as vertices is created, the volume of the rectangular parallelepiped MB is calculated, and the calculated value is set as a measured value of the volume to be measured.

Next, the operation will be described. FIG. 10 is a flowchart illustrating a procedure of a process according to the fourth embodiment. First, in step ST40, the display partial area designation operation unit 10 responds to the contents of the measurement on the display screen of the display device 3 on which the captured image is displayed, as shown in FIGS. Input operation for measurement. Next, in step ST41, the model partial region calculation unit 14 creates a model partial region such as a straight line or a three-dimensional object according to the input data from the display partial region designation operation unit 10, and checks for interference between the model partial region and the monitoring target model. Then, in step ST42, it is determined whether or not there is a monitoring target model that causes interference.

As a result, if there is no monitoring target model that causes interference, the monitoring worker is notified in step ST43 that there is no monitoring target object. On the other hand, if there are two or more monitoring targets that cause interference, the process proceeds to step ST44, and the measurement target surface is sorted according to a predetermined rule, as in the second embodiment, and the process proceeds to step ST4.
At 5, the contents are presented to the monitoring operator. The monitoring operator selects a required monitoring target surface in step ST46, and the model partial area calculation unit 14 calculates a distance, an area, and the like according to the measurement content based on the selected monitoring target surface in step ST47.

As described above, according to the fourth embodiment, the measurement parameters specified on the display screen on which the captured image is displayed are converted into the measurement parameters in the three-dimensional model, and the actual site is simulated. Since various measurements can be performed using a three-dimensional model, the measurement time can be shortened by installing a measurement device in the realization place, and the actual calculation of the area such as water leakage and oil leakage can be performed. Even when the measurement is complicated and difficult, the measurement can be performed immediately from the captured image, and the measurement time can be greatly reduced.

Embodiment 5 In the fifth embodiment, the displayed monitoring target is expressed as a two-dimensional figure projected onto one curved surface, thereby reducing the data amount required for model management and the processing time required for region extraction. . Note that the configuration of the device is basically the same as that of the first embodiment shown in FIG. 1, and a description thereof will be omitted.

FIG. 11 is an explanatory diagram showing a surface model and a model projection area according to the fifth embodiment of the present invention. In FIG. 11, when a surface model is used as the imaging target model 13 in the first embodiment, the surface model and the imaging projection plane calculated using the imaging parameters are projected into the surface model. FIG. 11A shows an example in which a plane is used as a surface model.
11B illustrates an example using a cylindrical surface, and FIG. 11C illustrates an example using a spherical surface. However, when the vertical and horizontal rotation angles of the imaging device 2 are smaller than 180 degrees, a plane model can be used. When the left and right rotation angles are larger than 180 degrees, a cylindrical surface model can be used. If the vertical rotation angle is also greater than 180 degrees, a spherical model can be used.

Next, the operation will be described. FIG. 12 is a flowchart illustrating a procedure of a process according to the fifth embodiment. First, in step ST50, the imaging projection plane is arranged in the model space of the plane model using the imaging parameters. Next, in step ST51, the model partial area calculation unit 14 uses the imaging partial area calculated by the imaging partial area calculation unit 12 from the contour shape and the viewpoint surrounding the imaging partial area in the imaging projection plane, and An outline figure on the surface model corresponding to the imaging partial area, which is an area, is created.

Then, the model extracting unit 15 executes step ST
In step 52, an interference check is performed between the contour figure as the model partial region and the monitoring target model in the imaging target model 13, and in step ST53, it is determined whether or not an interference monitoring target model is detected by the interference check. I do. As a result, if there is no interfering monitoring target model, the fact is notified to the monitoring operator in step ST54, and the process is terminated. On the other hand, if there are two or more monitoring target models that interfere with each other, the process proceeds to step ST55,
Sorting is performed according to a rule that is appropriate for the operation, such as from a viewpoint close to the viewpoint or from a viewpoint with a high degree of interference. In step ST56, the contents of the sorting are displayed on the display device 3 to notify the monitoring operator. In step ST57, the monitoring operator selects an appropriate monitoring target from among them. If there is one monitoring target model that interferes, that one is automatically selected.

As described above, according to the fifth embodiment, the monitored object displayed as the captured image as the captured object model is expressed as a two-dimensional figure projected on one surface. The amount of data required for management can be reduced, the load on the area extraction processing can be reduced, and a high-speed response can be obtained even with a device having low processing capacity.

Embodiment 6 FIG. The sixth embodiment relates to a method of creating a monitoring target model in an imaging target model in a monitoring device using a captured image according to the present invention. The configuration of the monitoring device using the captured image is the same as that of the embodiment shown in FIG. Since this is the same as in the case of the first embodiment, the description is omitted here. FIG. 13 is a flowchart showing a processing procedure of a model creating method according to Embodiment 6 of the present invention. In addition,
FIG. 13 shows an example of a process for creating a surface model using a captured image, which is a process for creating a partial model region of a surface model using a partial region in the fifth embodiment. It becomes the same.

Next, the operation will be described. First, when the process is started in step ST60, the process proceeds to step ST6.
At 1, the captured image is displayed on the display device 3, and the captured image displayed on the display screen is used to trace the image of the monitoring target object for which the monitoring target model is to be created using the display partial area instruction operation unit 10, Alternatively, the outline of the monitored object is created on the display screen by a method such as automatically extracting the outline of the monitored object by image processing or the like. Next, in step ST62, the outline of the monitoring target created in this manner is converted into an outline on the imaging projection plane, and further converted into an outline figure of a surface model which is a monitoring target model. Next, in step ST63, the converted figure thus obtained is registered as a monitoring target model together with the identifier of the monitoring target input by the creator to create the imaging target model 13. Is completed.

As described above, according to the sixth embodiment, while observing the display screen on which the captured image is displayed, the captured image of the monitored object requiring model creation is displayed. It is possible to create a monitored model simply by directly using its outline to create a model to be monitored, reducing the time required to create the model and eliminating the need for special training for the model creator. And so on.

Embodiment 7 FIG. The seventh embodiment relates to a model creation method for notifying a creator of the presence / absence of creation of a monitoring target model. FIG. 14 is an explanatory diagram showing an example of processing of a model creation method according to a seventh embodiment of the present invention. is there. FIG. 14A shows an example in which a captured image of a monitoring target is displayed on the display device 3, in which 50 is a display screen, 51 to 53 are display images of a model-created monitoring target, and 54. Reference numerals 56 are display images of the monitoring target object for which no model has been created. FIGS. 14B to 14D show examples in which the created monitoring target models are displayed.
In FIG. 14B, reference numerals 57 to 59 denote the monitored model superimposed on the captured image using the created model shape, and reference numerals 60 to 62 in FIG. Marks displayed, 6 in FIG. 14 (d)
Reference numerals 3 to 65 indicate the identifiers of the monitoring objects in the center of the created model shape. FIG. 14E shows FIG.
This is an example in which the imaging parameters of the imaging device are changed during model creation as shown in (b). Reference numeral 66 in the drawing denotes a display screen, which corresponds to the fact that the imaging parameters have changed and the captured image has moved to the left. This shows that the display image of the monitoring target for which the model has been created is also moving to the left.

Next, the operation will be described. FIG. 15 is a flowchart showing the procedure of processing when notifying the presence / absence of creation of a monitoring target model on a display screen on which a captured image is displayed from the imaging target model 13 in the seventh embodiment. If the imaging parameter does not change, step S
In T70, a model creation operation is performed to create a model figure, and the monitoring target model is registered. Next, step S
At T71, the model shapes 57 to 5 shown in FIG.
9, the circular marks 60 to 62 shown in FIG. 14C or the monitoring target identifiers 63 to 65 shown in FIG.
Create graphical information such as Next, in step ST72, a position on the display screen is obtained from the position of each monitoring target model in the imaging target model 13, and the created graphical information is obtained as shown in FIGS. 14 (b) to 14 (d). The image is superimposed on the captured image corresponding to the monitoring target model.

On the other hand, when the imaging parameters change, if the imaging parameters of the imaging device 2 are changed in step ST73, the display image displayed on the display screen of the display device 3 is changed in step ST74. It changes according to the change. Next, in step ST75, a model partial region of the surface model is calculated by performing the extraction processing described in the fifth embodiment on the entire region of the display screen displaying the captured image. And step ST
At 76, all the monitoring target models registered in the model partial area are extracted, and the process proceeds to step ST71 to create graphical information to be displayed on the display device 3 from each model figure, and the created graphical information is created at step ST72. The graphical information is displayed so as to overlap the captured image corresponding to the monitoring target model.

As described above, according to the seventh embodiment, the graphical information indicating that the model creation has been completed is displayed so as to overlap the captured image of the monitored object. By observing the captured image on the screen, it is possible to recognize a monitoring target object for which model creation has not been completed, and to prevent double creation of the monitoring target model or omission of model creation.

Embodiment 8 FIG. In the eighth embodiment, the monitoring information is displayed so as to be superimposed on the captured image at the display position of the monitoring target model. The device configuration is basically the same as that of the first embodiment shown in FIG. Since they are the same, the description is omitted here.

FIG. 16 is an explanatory diagram showing a display example of a monitoring device using captured images according to the eighth embodiment of the present invention.
FIG. 16 shows an example in which the display position of the control information on the display screen on which the captured image is displayed is obtained from the monitoring target model in the imaging target model 13. In FIG. This is a window in which the output data is displayed. FIG. 16A shows the center position of the monitoring target model of the monitoring target object 70 shown in the captured image as a point of interest, the position of the point is converted to a display screen, and the position is output as monitoring data output data. FIG. 16B shows an example in which the upper right position of the monitoring target model of the monitoring target object 70 is converted to a display screen, and the position is displayed at the upper left position of the window 71. The example shown in FIG.

Next, the operation will be described. Note that a method of converting the position of the monitoring target model in the imaging target model 13 into a point on the display screen on which the captured image is displayed is the same as the processing in the seventh embodiment, and thus the description thereof is omitted here. .

FIG. 17 is a flowchart showing the procedure of the process in the eighth embodiment. First, in step ST80, the monitoring information output processing unit 16 extracts a monitoring target model corresponding to the input monitoring target object identifier. Next, in step ST81, a point of interest is obtained from a preset point of the monitoring target model, a point created from the shape data of the monitoring target model, and the like. Next, in step ST82, the point of interest is converted into a point on the display screen on which the captured image is displayed, and the coordinates of the position are calculated. Next, in step ST83, FIG.
As shown in FIG. 16A or FIG. 16B, the monitoring information is displayed at the calculated position on the display screen so as to overlap the captured image.

In the above description, the monitoring operator inputs the identifier of the monitoring target, and the monitoring information output processing unit 16 extracts the monitoring target model from the imaging target model 13 based on the input. For example, when the monitoring operator instructs the monitoring target of the captured image on the display screen of the display device 3, the monitoring information output processing unit 16 recognizes the monitoring target and changes the monitoring target model in the imaging target model 13. You may make it extract.

Further, the monitoring information output processing section 16 detects the presence or absence of an abnormality or a failure of the monitoring target at the actual site, and based on the detection result, the imaging target model 13.
The monitoring target model is extracted from among the information, and information such as an abnormality notification or warning for a device in which an abnormality or a failure detected by the monitoring information output processing unit 16 has been detected is superimposed on the captured image as monitoring information. You may.

As described above, according to the eighth embodiment, since the monitoring information of a monitoring target is displayed on the same display screen as the captured image of the monitoring target, the monitoring information It is possible to instantly recognize whether the data is the data of the monitoring target, and it is possible to prevent a mistake in the monitoring work and to improve the responsiveness from the monitoring work to the next work. In addition, if the monitoring operator specifies the target monitoring target on the display screen, the monitoring information can be displayed on the captured image of the monitoring target, and the monitoring information can be displayed without moving the viewpoint after the selection. It is also possible to automatically display control information according to the situation of the site on the display screen on which the captured image is displayed, so that the control information can be automatically superimposed on the monitoring target in which the situation has occurred.
Even if an abnormality or failure occurs, it is possible to instantly recognize what kind of situation has occurred on which monitoring target, and to improve the responsiveness from monitoring work to the next work. is there.

Embodiment 9 The ninth embodiment relates to a control method of an imaging device in a monitoring device using a captured image according to the present invention. Note that the configuration of the monitoring device using the captured image is the same as that of the first embodiment shown in FIG. 1, and a description thereof will be omitted here. FIG. 18 is an explanatory diagram showing processing of an imaging device control method according to Embodiment 9 of the present invention. In the figure, reference numeral 80 denotes a monitoring target model among the imaging target models.

FIG. 18A illustrates the monitoring target model and the line of sight of the image pickup device 2. Imaging target model 1
3 from the identifier of the monitoring target model 80 to its attention point M
P is obtained, and a direction vector PV to the target point MP in the coordinate system of the imaging device 2 is obtained from the imaging parameters. Then, the driving control parameters of the imaging device 2 are calculated so that the front of the imaging device 2 faces the direction vector PV, and the imaging device 2 is rotated. FIG. 18B shows the coordinate system of the imaging device 2, where CX is the coordinates in the left and right viewing directions, and CY is the coordinates in the up and down viewing directions. The direction vector PV in this case is -θx in the horizontal direction and + θ in the vertical direction.
in the direction of y.

FIG. 19 is an explanatory diagram showing an example of a change in a picked-up image when the image pickup device 2 according to the ninth embodiment is rotated toward a monitored object to be watched. Reference numeral 81 in FIG. Monitoring object to be monitored. Note that FIG. 19A illustrates a captured image before the image capturing device 2 is rotated, and FIG. 19B illustrates a captured image after the image capturing device 2 is rotated.

Next, the operation will be described. FIG. 20 is a flowchart illustrating a processing procedure of an imaging device control method according to the ninth embodiment. First, in step ST90,
Based on the identifier of the monitoring target 81 input by the monitoring worker, the corresponding monitoring target model 80 in the imaging target model 13 is extracted. Next, in step ST91, a point of interest MP is extracted from a preset point of the extracted monitoring target model 80 or a point created from the shape data of the monitoring target model 80. Next, in step ST92, the target point M of the monitoring target model 80 is set.
The direction of P is converted into a direction vector PV in the coordinate system CX, CY of the imaging device 2, and in step ST93, a drive control parameter of the imaging device 2 is calculated so that the direction vector PV is directly in front of the imaging device 2. .

Then, it is determined in step ST94 whether or not the drive control parameter is within the allowable range of the drive area of the imaging device 2. As a result, if it falls within the allowable range, the process proceeds to step ST95, and the imaging device 2 is automatically controlled to face the designated monitoring target 81 according to the drive control parameters. On the other hand, if not within the allowable range, in step ST96,
After obtaining the parameter closest to the target parameter within the allowable range of the drive area of the imaging device 2 and changing the drive control parameter to the approximate parameter, the process proceeds to step ST95, and the imaging device 2 is operated in accordance with the changed drive control parameter. Control. As a result, the monitoring target 81 that has deviated from the captured image as shown in FIG.
It can be moved to the center of the captured image as shown in FIG.

In the above description, the monitoring target model 80 is extracted from the imaging target model 13 based on the identifier of the monitoring target 81 input by the monitoring worker. May indicate the monitoring target 81 of the captured image on the display screen of the display device 3, and the monitoring target model 80 may be extracted by recognizing the instruction.

The monitoring information output processing unit 16 detects whether an abnormality or a failure has occurred in the monitoring target 81 at the actual site, and based on the detection result, the monitoring target model 80 in the imaging target model 13. May be extracted.

As described above, according to the ninth embodiment, just by giving the identifier of the object to be watched, the imaging device automatically turns in that direction, and the object to be monitored moves to the center of the captured image. As a result, the monitoring worker can catch the monitoring target that he / she wants to pay attention to as an imaging target without operating the imaging device, shorten the time for operating the imaging device, and perform the next operation from the monitoring operation. There are effects such as being able to improve responsiveness to work. Further, when the monitoring target model is extracted by designating a monitoring target to be focused on a display screen on which a captured image is displayed, the monitoring target model can be monitored without knowing the identifier of the monitoring target. When the target object can be regarded as an imaging target and the monitoring target model is extracted based on the detection result of the abnormality or failure of the monitoring target at the actual site, the monitoring worker Is effective in that a monitoring target in which an abnormality or the like has occurred can be automatically captured as an imaging target without operating the imaging device.

Embodiment 10 FIG. This Embodiment 10
After the imaging device 2 is controlled by the imaging device control method described in the ninth embodiment so that the monitoring target object of interest is located at the center of the imaging screen, the monitoring target model in the imaging target model is further displayed on the display device 3. The zoom-in and zoom-out of the imaging device 2 are controlled so that the image is displayed on the entire display screen. FIG. 21 is an explanatory diagram showing processing of an imaging device control method according to Embodiment 10 of the present invention. In the figure, 90 is an imaging target model, 91 and 92 are imaging regions, and 93 is an imaging target model 90. A monitoring target model of interest, 94 is a captured image projected on the display device 3, and 95 is a captured image of the monitoring target.

FIG. 21A shows a monitoring target model 93 to be observed in an imaging target model 90 and two imaging regions 91 and 92 including the monitoring target model 93. FIG. 21B is an example in which a captured image 95 of the monitoring target when the imaging region is 91 is shown as the captured image 94 shown on the display device 3.
(C) is an example showing a captured image 95 of the monitoring target when a minimum two-dimensional figure surrounding the monitoring target model 93 is set in a shape similar to the imaging region 91 as the imaging region 92.

Next, the operation will be described. First, the rotation control of the imaging apparatus 2 described in the ninth embodiment is performed so that the monitoring target object of interest comes to the center of the imaging screen. Thereafter, a figure of the entire region of the captured image displayed on the display device 3 is created for a certain imaging parameter, and the captured projected figure obtained by projecting the extracted monitoring target model 93 on the captured projection plane is displayed in the figure. A minimum two-dimensional figure is created among figures surrounded by similar figures. In addition, for the creation of the minimum two-dimensional figure, for example, processing realized by many two-dimensional CAD, figure creation software, or the like is used. Then, a drive control parameter for zooming in and out of the imaging device 2 is created so that the minimum two-dimensional figure becomes the entire region of the captured image, and the generated drive control parameter is provided to the drive control unit of the imaging device 2 together with a drive command. This also controls zoom-in and zoom-out of the imaging device 2.

Here, FIG. 22 is a flowchart showing a procedure of processing after rotation control of the imaging device 2 according to the tenth embodiment. First, in step ST100, a monitoring target model 93 at the center of the imaging region 91 (92) is extracted. Next, in step ST101, a two-dimensional figure of the imaging region 91 (92) is obtained from the shape of the monitoring target model 93. For example, if the shape of the imaging region 91 (92) is a rectangle, its vertical and horizontal sizes are obtained. Next, in step ST102, the smallest two-dimensional figure surrounding the projected figure of the monitoring target model 93 on the image projected plane is created with a similar figure of the imaging area 91 (92). For example, if the shape of the imaging area 91 (92) is rectangular, a minimum two-dimensional figure surrounding the projected figure is created with a two-dimensional figure (rectangle) having the same aspect ratio. Further, in step ST103,
A drive control parameter for zooming in or zooming out the imaging apparatus 2 is calculated so that the area of the minimum two-dimensional figure becomes the imaging area 91 (92).

Then, it is determined in step ST104 whether or not the drive control parameter is within the allowable range of the drive area of the imaging device 2. As a result, if it falls within the allowable range, the process proceeds to step ST105, and the imaging device 2 is automatically controlled according to the drive control parameters so as to face the designated monitoring target. On the other hand, if it is not within the allowable range, in step ST106, a parameter closest to the target parameter within the allowable range of the driving area of the imaging device 2 is obtained, and the drive control parameter is changed to the approximate parameter, and then, in step ST10.
Proceeding to 5, the zoom-in and zoom-out of the imaging device 2 are controlled according to the changed drive control parameters. Thereby, as shown in FIG. 21C, the captured image 95 of the monitoring target can be displayed so as to be the largest in the captured image 94 reflected on the display device 3.

As described above, according to the tenth embodiment, the object to be watched is controlled to zoom in and out of the image pickup apparatus so that the object to be monitored has the maximum size that can fit in the captured image. Can be performed automatically, and the monitoring worker can view the monitoring target as an imaging target in an appropriate size without operating the imaging device, reducing the time required for operating the imaging device. This makes it possible to improve the responsiveness from the monitoring operation to the next operation.

[0088]

As described above, according to the first aspect of the present invention, there is provided an imaging target model obtained by modeling an actual site, and a display partial area in a display screen of a display device is designated and displayed. Obtain the imaging partial area of the imaging device from the partial area, extract the monitoring target model in the model partial area from the obtained imaging partial area and the imaging target model, and recognize the actual target monitoring target from the monitoring target model. Then, since the configuration is such that the information of the monitoring target is displayed on the display device, it is possible to extract the monitoring target model from the imaging target model corresponding to the captured image using the partial region, and Even when there is an error between the captured image and the captured image corresponding to the captured image, it is possible to reliably extract the monitored object model, and there is an effect that it does not take much time to select the monitored object.

According to the second aspect of the present invention, the creation of the model partial area is performed using the input imaging parameters and the imaging partial area of the imaging device, and the monitoring target model overlapping the model partial area is set as the monitoring target. Since it is configured to extract, the monitoring target model can be reliably extracted even if an error occurs between the captured image and the imaging target model.

According to the third aspect of the present invention, a three-dimensional model is used as a model sub-region, a three-dimensional model as a model sub-region created by a model sub-region calculation unit, and a three-dimensional model of a monitored object. Is performed by the model extraction unit, and the monitoring target model is extracted based on the result of the interference check. Therefore, it is possible to specify a monitoring target that does not appear in the captured image, and Since the range of the target can be greatly expanded, and even when the position of the imaging device is changed, the system can be easily reconfigured only by changing the arrangement position of the imaging device in the imaging parameters, There is an effect that it is possible to flexibly respond to a change in the system and to easily add an imaging device.

According to the fourth aspect of the present invention, when a plurality of monitored models interfere with the three-dimensional partial model, the monitored models are sorted according to a predetermined rule and displayed. When there are two or more monitoring target models that interfere with each other, there is an effect that the monitoring operator can select and extract an optimum monitoring target model from the monitoring target models.

According to the fifth aspect of the present invention, since the three-dimensional model as the imaging target model can be changed at any time based on the actual state of the site, the actual monitoring target moving the three-dimensional model can be changed. Can be changed so as to simulate the movement of the target, so that the actual site and the imaging target model can be the same, and dynamically changing monitoring targets can be selected, limiting the types of monitoring targets There is an effect that a surveillance operation that is not possible can be performed.

According to the sixth aspect of the present invention, the designation information for designating the measurement part input from the display partial area designation operation unit is converted into the model partial area by the model partial area calculation unit.
Since the measurement calculation in the three-dimensional model is performed, various measurements can be performed using the captured image displayed on the display device, and the time for installing and measuring the measurement device can be shortened, and water can be reduced. Even when the actual measurement is complicated and difficult such as calculation of the area such as leakage or oil leakage, the measurement can be performed immediately using the captured image, and the measurement time can be shortened.

According to the seventh aspect of the present invention, a surface model is used as an imaging target model, and a model partial area corresponding to the imaging partial area is determined based on the imaging partial area calculated by the imaging partial area calculation unit. Since the monitoring target object is extracted by obtaining a monitoring target model that interferes with the created model partial region, a surface model according to the rotation capability of the imaging device is used as the imaging target model. It is possible to reduce the amount of management data of the model and to extract the monitoring target model by two-dimensional processing, so that the load of the extraction processing can be reduced,
There is an effect that a high-speed response can be obtained even with a device having a low calculation capability.

According to the eighth aspect of the present invention, the outline of the monitored object is obtained by directly specifying the monitored object projected on the display screen, and the obtained outline is converted into the outline on the imaging projection plane. After the conversion, it is further configured to create a monitoring target model by converting it to a contour figure of the monitoring target model, so that it is possible to create an imaging target model using a captured image projected on the display device, There is no need to provide special training to the monitoring workers for model creation, and a model creation method that can create an imaging target model with simple operations and can reduce the model creation time can be obtained. effective.

According to the ninth aspect of the present invention, the position on the display device corresponding to the position of each monitoring target model is calculated, and the graphical information indicating that the monitoring target model has been created is displayed. Since the target model is configured to be superimposed on the captured image at the position on the display device, which monitoring target model has been created and which monitoring target model has not been created However, since it can be recognized on the display screen of the captured image which is the monitoring target model creation screen, there is an effect that double creation or omission of creation of the monitoring target model can be prevented.

According to the tenth aspect, the point of interest obtained from the monitoring target model extracted from the imaging target model is converted into a point on the display screen on which the captured image is displayed, and the point corresponding to the position is converted. The monitoring information of the monitored object to be monitored is superimposed on the captured image, so that the monitored information and the monitored object are displayed on the same screen, and the response is instantaneous. It is possible to prevent inconsistency in correspondence between objects, prevent mistakes in monitoring work, and improve responsiveness from monitoring operation to action to the next operation.

According to the eleventh aspect of the present invention, the point of interest obtained from the monitoring target model extracted from the imaging target model is converted into a direction vector in the coordinate system of the imaging device, and the imaging device is converted into the direction vector of the direction vector. It is configured to control the rotation of the imaging device by calculating the drive control parameter for pointing in the direction, so it is possible to automatically point the imaging device in the direction of the monitoring target that you want to focus on and catch the monitoring target Thus, there is an effect that the operation time of the imaging device can be reduced.

According to the twelfth aspect of the present invention, the monitoring target model in the imaging target model is configured to be extracted based on the identifier of the input monitoring target object. There is an effect that the direction of the imaging apparatus can be automatically controlled so that the monitoring target is located at the center of the captured image only by giving the monitoring target identifier without knowing the arrangement information of the monitoring target.

According to the thirteenth aspect of the present invention, the monitoring target model in the imaging target model is extracted based on the position on the display screen of the monitoring target specified by the monitoring operator. By simply designating the monitoring target in the captured image on the display screen, the direction of the imaging device can be automatically controlled so that the monitoring target is located at the center of the captured image.

According to the fourteenth aspect of the present invention, the monitoring target model in the imaging target model is extracted based on the result of detecting whether an abnormality or a failure has occurred in the monitoring target at the actual site. Therefore, when an abnormality or a failure occurs in the monitored object, the direction of the imaging device can be automatically controlled so that the monitored object is located at the center of the captured image.

According to the fifteenth aspect of the present invention, the image pickup device is zoomed in or out so that the smallest two-dimensional figure surrounding the extracted projected figure of the monitoring target model coincides with the image pickup area of the image pickup apparatus. The drive control parameters are calculated, and zoom-in and zoom-out control of the imaging device are performed based on the drive control parameters, so that the imaging range of the imaging device can be automatically enlarged or reduced according to the size of the monitoring target. Since the control can be performed, not only the operation time of the imaging apparatus can be reduced, but also the responsiveness from the monitoring operation to the action to the next operation can be improved.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a monitoring device using captured images according to Embodiment 1 of the present invention.

FIG. 2 is an explanatory diagram for describing processing according to the first embodiment of the present invention.

FIG. 3 is a flowchart showing a procedure of a process according to the first embodiment of the present invention.

FIG. 4 is a schematic diagram illustrating an imaging target model used in a monitoring device using a captured image according to Embodiment 2 of the present invention;

FIG. 5 is a schematic diagram showing a model partial region according to Embodiment 2 of the present invention.

FIG. 6 is a flowchart showing a procedure of a process according to the second embodiment of the present invention.

FIG. 7 is a block diagram showing a configuration of a monitoring device using captured images according to Embodiment 3 of the present invention.

FIG. 8 is a flowchart showing a procedure of processing according to Embodiment 3 of the present invention.

FIG. 9 is an explanatory diagram illustrating measurement processing by a monitoring device using a captured image according to Embodiment 4 of the present invention;

FIG. 10 is a flowchart showing a procedure of processing according to Embodiment 4 of the present invention.

FIG. 11 is an explanatory diagram showing a surface model and a model projection region in a monitoring device using captured images according to a fifth embodiment of the present invention.

FIG. 12 is a flowchart showing a procedure of processing according to Embodiment 5 of the present invention.

FIG. 13 is a flowchart showing a procedure of processing in a model creation method according to Embodiment 6 of the present invention.

FIG. 14 is an explanatory diagram showing a process of a model creation method according to a seventh embodiment of the present invention.

FIG. 15 is a flowchart showing a procedure of processing according to Embodiment 7 of the present invention.

FIG. 16 is an explanatory diagram illustrating a display example of a monitoring device using captured images according to Embodiment 8 of the present invention;

FIG. 17 is a flowchart illustrating a procedure of processing according to the eighth embodiment of the present invention.

FIG. 18 is an explanatory diagram illustrating processing of an imaging device control method according to Embodiment 9 of the present invention;

FIG. 19 is an explanatory diagram showing changes in a captured image according to Embodiment 9 of the present invention.

FIG. 20 is a flowchart showing a procedure of processing according to Embodiment 9 of the present invention.

FIG. 21 is an explanatory diagram illustrating processing of an imaging device control method according to Embodiment 10 of the present invention;

FIG. 22 is a flowchart showing a procedure of processing according to Embodiment 10 of the present invention.

FIG. 23 is a block diagram showing a configuration of a conventional monitoring device using captured images.

FIG. 24 is a flowchart illustrating a processing procedure of a monitoring apparatus using a conventional captured image.

[Explanation of symbols]

2 imaging device, 3 display device, 10 display partial region instruction operation unit, 11 display control processing unit, 12 imaging partial region calculation unit, 13 imaging target model, 14 model partial region calculation unit, 15 model extraction unit, 16 monitoring information output Processing unit.

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) H04N 7/18 G05B 23/02 G06T 1/00

Claims (15)

(57) [Claims] An imaging device that captures an image of an actual site to be monitored; a display device that displays an image captured by the imaging device; a display control processing unit that controls display on the display device; An imaging target model that models a site, a display partial area instruction operation unit for instructing a display partial area in a display screen of the display device, and a display partial area instructed by the display partial area instruction operation unit An imaging partial area calculation unit that obtains an imaging partial area of the imaging device; a model partial area calculation unit that calculates a model partial area from the imaging partial area obtained by the imaging partial area calculation unit and the imaging target model; A model extraction unit for extracting a monitoring target model in the model partial region calculated by the model partial region calculation unit; Recognizes the monitoring target in the field, the monitoring device using the captured image and a monitor information output processing unit for displaying the information of the monitored object on the display device via the display control unit. 2. A model partial area calculation unit receives an imaging parameter of an imaging device together with an imaging partial area obtained by the imaging partial area calculation unit, and uses the imaging parameters to change an imaging projection plane of an imaging target model. A model extracting unit that has a function of extracting the monitoring target model overlapping the model partial region created by the model partial region calculating unit as a monitoring target; The monitoring device according to claim 1, wherein the monitoring device uses a captured image. 3. A three-dimensional model is used as an imaging target model that models an actual site, and a model partial area calculation unit arranges the imaging device in the three-dimensional model at the same position and orientation as the actual site. And a function of creating a three-dimensional partial model of the imaging region using the imaging partial region calculated by the imaging partial region calculation unit. One having a function of calculating interference between a partial model and a monitoring target model which is a three-dimensional model of the monitoring target, and extracting the monitoring target model in which interference has occurred between the three-dimensional partial model as a monitoring target. The monitoring device using a captured image according to claim 1 or 2, wherein: 4. When there are two or more monitoring target models that interfere with the three-dimensional partial model created by the model partial region calculating unit, the model extraction unit determines a monitoring target model that interferes with the three-dimensional partial model. 4. The monitoring apparatus according to claim 3, wherein the monitoring apparatus has a function of performing sorting according to the following rule and displaying the result on a display device. 5. The monitoring information output processing section has a function of changing a three-dimensional model as an imaging target model that models the actual site based on the state of the actual site. A monitoring device using the captured image according to claim 3. 6. A display partial area instruction operation unit having a function of inputting designation information for designating a part to be measured with respect to a captured image projected on a display device, wherein the model partial area calculation unit 4. The apparatus according to claim 3, further comprising a function of converting designation information input from the display partial area instruction operation unit into a model partial area and performing measurement calculation in a three-dimensional model. 5. A monitoring device using the captured image according to any one of 5. 7. A plane model is used as an imaging target model obtained by modeling an actual site, and a model partial area calculation unit corresponds to an imaging partial area using the imaging partial area calculated by the imaging partial area calculation unit. A model extracting unit that has a function of creating a model partial area on the surface model, wherein a model extracting unit obtains a monitoring target model that interferes with the model partial area created by the model partial area calculating unit, 2. The monitoring apparatus using a captured image according to claim 1, wherein the monitoring apparatus extracts the image as a target. 8. An imaging target model for associating a captured image with an actual site to be monitored, wherein correspondence with the imaging target model is obtained by using imaging parameters of an imaging device, From the partial area on the display screen of the display device that displays the image, calculates the imaging partial area of the imaging projection plane, obtains the model partial area of the imaging target model, and extracts the monitoring target model overlapping the model partial area In a model creation method for creating a monitoring target model in the imaging target model, a monitoring device using the above, by directly indicating the monitoring target projected on the display screen of the display device, the monitoring target Create a contour, convert the obtained contour into a contour on the imaging projection plane, and convert it to a contour figure of the monitor model to create a monitored model. Model creation method comprising the door. 9. A diagram that calculates a position on the display device corresponding to a position of each monitoring target model in the imaging target model, and indicates that the monitoring target model has been created for the created monitoring target model. 10. The model creation method according to claim 9, wherein target information is created, and the captured image is superimposed and displayed at a position on the display device corresponding to the position of the monitoring target model. 10. A monitoring information output processing unit extracts a monitoring target model from an imaging target model, obtains a point of interest of the monitoring target model, and determines the point of interest on a display screen on which a captured image is displayed. 8. A function of superimposing the captured image on the position and displaying the monitoring information of the corresponding monitoring target object at the position. A monitoring device using the captured image according to claim 1. 11. An imaging target model for associating a captured image with an actual site to be monitored, wherein the imaging device uses an imaging parameter of an imaging device to associate with the imaging target model, and From the partial area on the display screen of the display device that displays the image, calculates the imaging partial area of the imaging projection plane, obtains the model partial area of the imaging target model, and extracts the monitoring target model overlapping the model partial area A monitoring device that controls the imaging device by using a monitoring device that extracts a monitoring target model from the imaging target model to obtain a point of interest of the monitoring target model; The drive control parameter for directing the imaging device in the direction of the obtained direction vector is calculated by converting the drive control parameter into a direction vector in the coordinate system of An imaging device control method, comprising controlling the imaging device based on parameters. 12. The method according to claim 11, wherein the extraction of the monitoring target model in the imaging target model is performed based on the input identifier of the monitoring target object. 13. A method of extracting a monitoring target model from an imaging target model, the monitoring target object being instructed by a monitoring worker,
2. The method according to claim 1, wherein the determination is performed based on a position on a display screen of the display device on which the captured image is displayed.
2. The imaging device control method according to claim 1. 14. Detection of the presence or absence of an abnormality or a failure with respect to the actual monitoring target at the site, and extraction of the monitoring target model in the imaging target model based on the detection result of the occurrence of the abnormality or the failure. The method according to claim 11, wherein the control is performed by performing the following. 15. A minimum two-dimensional figure surrounding the projected figure is obtained based on the shape of the projected figure of the monitoring target model extracted from the imaging target model, and the obtained minimum two-dimensional figure is used for the imaging apparatus. Calculating drive control parameters for controlling zoom-in and zoom-out of the imaging device so as to match the entire region of the captured image; and controlling the imaging device based on the obtained drive control parameters. The imaging device control method according to any one of claims 11 to 14, wherein: