JP5960471B2 - Image monitoring device - Google Patents

Description

  The present invention relates to an image monitoring apparatus that monitors an abnormal state at a monitoring place by performing image processing on images taken at the monitoring place taken by a plurality of monitoring cameras.

  In recent years, image monitoring apparatuses that centrally monitor abnormal states in a plurality of areas at a monitoring location by sequentially receiving captured images captured by a plurality of monitoring cameras and displaying them on a monitoring terminal used by a supervisor are used. ing. Conventionally, in such an image monitoring apparatus using a plurality of monitoring cameras, a sensor installed at a monitoring location and the monitoring camera are stored in association with each other, and when an abnormal signal is received from the sensor, the sensor corresponds to the sensor. A technique for centrally monitoring abnormal states in a plurality of areas at a monitoring location by displaying a captured image of the monitoring camera on a monitoring terminal has been used (for example, Patent Document 1).

JP-A-10-188183

  However, with the rapid increase in the number of installed surveillance cameras as in recent years, it has been difficult for the supervisor to properly monitor the abnormal state with the monitoring method as in the prior art. That is, the above-described conventional technique is effective when monitoring an abnormal state with a low occurrence frequency, but when monitoring an abnormal state with a high occurrence frequency, display switching of captured images frequently occurs. There is a problem that it is difficult to appropriately monitor the abnormal state. Therefore, attribute information that can distinguish between an object that should be monitored with priority and an object that is not so with respect to an object at a monitoring location is added, and the attribute information can be dynamically changed according to the situation at the monitoring location. Therefore, there is a need to limit the monitoring target and monitor the abnormal state. In particular, there is a need to allow a monitored person at a monitoring location to dynamically change object attribute information as necessary.

  Therefore, the present invention can detect the change operation of the attribute information by the monitored person by the monitoring camera, and change the attribute information of the monitoring target object according to the spatial positional relationship between the position where the change operation is performed and the object. Thus, an object is to satisfy the above-described needs with a simple mechanism, and to perform monitoring without increasing the monitoring load even in an environment where a large number of cameras are installed.

In order to achieve the above-described object, an image monitoring apparatus according to claim 1 of the present application includes:
An imaging unit that sequentially acquires captured images of the monitoring location;
Expressed by a location model that represents the monitoring location as a three-dimensional virtual space, a three-dimensional shape of an object that is a component of the location model and exists at the monitoring location, and a virtual space position that is a position in the virtual space A storage unit that stores an object table that associates the acquired object model with attribute information of the object model, and imaging condition information that associates the installation position, optical axis, and angle of view of the imaging unit with the virtual space; ,
Attention change area detection means for extracting a change area from the captured image and detecting the change area as an attention change area when the change area has a predetermined image feature;
Virtual space position calculation means for obtaining a virtual space position of the target change area using the location model, the imaging condition information, and a position of the target change area on the captured image;
Attribute changing means for changing the attribute information of the object model having a predetermined spatial positional relationship with the attention change area in the virtual space, using the virtual space position of the attention change area and the virtual space position of the object model When,
And determining means for determining a state of the object from the captured image using the object table and outputting a determination result.

  With this configuration, the attention changing area detection unit of the present invention extracts a changing area from a captured image acquired from a plurality of imaging units installed at a monitoring location using a conventional technique such as background difference. Then, the attention change area detection means detects a change area (attention change area) that should be noted by the supervisor based on comparison with the preset image feature from the extracted change areas. And the virtual space position calculation means of this invention calculates | requires the position (virtual space position) in the virtual space of the detected attention change area. Then, the attribute changing means of the present invention determines whether the attention changing area and the object model are predetermined from the virtual space position of the attention changing area and the virtual space position of the object model stored in the storage unit as a component of the place model. When there is a spatial positional relationship, the attribute information of the object model is changed to, for example, a priority monitoring attribute that is attribute information regarding an object to be monitored mainly. Then, the determination means of the present invention determines, for example, that the intruder has changed area around the object whose attribute information in the object table is the priority monitoring attribute, and determines the determination result. For example, it is displayed on the monitoring terminal of the supervisor. With this configuration, the attention changing area detection unit of the present invention extracts a changing area from a captured image acquired from a plurality of imaging units installed at a monitoring location using a conventional technique such as background difference.

  Thereby, the image monitoring apparatus of the present invention images, for example, a mark (hereinafter, referred to as “marker”) set by a monitored person or the like at a monitoring place by the imaging unit, and the image characteristics of the marker from the captured image. When the change area of interest is detected as the attention change area, and the attention change area exists above the object, the attribute information of the object can be changed to the priority monitoring attribute. In other words, the person to be monitored can easily change the attribute information of the object simply by placing a marker on the object to be changed to the priority monitoring attribute. Therefore, the monitor can perform the monitoring operation by limiting the monitoring target to the object to be monitored with such a simple mechanism, even in an environment where a large number of cameras are installed. It is possible to monitor without increasing the monitoring load on the person.

The image monitoring apparatus according to claim 2 of the present application is the image monitoring apparatus of claim 1,
The storage unit further stores an attribute table in which an image feature of the attention changing area is associated with an attribute type of the attribute information, and the attribute changing unit is configured to store the image feature of the attribute table and the image of the attention changing area. The attribute type of the attention change area is determined by comparison with a feature, and the attribute information of the object table is changed according to the attribute type.

  With this configuration, the storage unit of the present invention stores in advance an attribute table in which different image features are associated with each attribute type of attribute information (for example, priority monitoring attribute, movement prohibition attribute, etc.). Then, when the extracted change area has any image feature of the attribute table, the attention change area detection unit of the present invention extracts the change area as the attention change area. Then, the attribute change means of the present invention compares the image feature of the attention change area detected by the attention change area detection means with the image feature of the attribute table, determines the attribute type of the attention change area, and The attribute information in the object table of the object model having a predetermined spatial positional relationship with the change area is changed according to the determined attribute type. For example, when the monitored person installs a marker representing the movement prohibition attribute on the object, the attribute information of the object model corresponding to the object is changed to the movement prohibition attribute. In this way, the monitored person can change the attribute information of the object in a plurality of ways according to the type of marker to be installed.

The image monitoring apparatus according to claim 3 of the present application is the image monitoring apparatus according to claim 1 or 2,
The virtual space position calculation means obtains the virtual space position of the change area using the location model, the imaging condition information, and the position of the change area on the picked-up image, and the determination means determines the object model. The state of the object corresponding to the object model is determined according to the attribute information and the positional relationship between the virtual space position and the virtual space position of the change area, and the determination result is output.

  With such a configuration, the virtual space position calculation means of the present invention obtains not only the virtual space position of the target change area but also the virtual space positions of all the change areas. The determination means of the present invention, for example, when another change region (for example, a person change region) exists within a predetermined range from the virtual space position of the object set as the priority monitoring attribute, The state is determined as an abnormal state. As described above, the image monitoring apparatus of the present invention takes into account the positional relationship between the attribute information of the object changed by the monitored person and the moving object (intruder) existing around the object. The abnormal state can be determined.

The image monitoring apparatus according to claim 4 of the present application is the image monitoring apparatus according to any one of claims 1 to 3,
The determination unit generates a rendering image corresponding to the field of view of the imaging unit, using the imaging condition information, for a place model where the object model is arranged, and an image region corresponding to the object model in the rendering image The movement of the object corresponding to the object model is determined by comparing the image feature with the image feature of the change area, and the attribute information of the object model corresponding to the object determined to have moved is prohibited from moving. In the case of the movement prohibition attribute, the determination result that makes the object in an abnormal state is output.

  With this configuration, the determination unit of the present invention determines the movement of the object that has been changed to the attribute (movement prohibition attribute) that prohibits the object movement by the attribute change unit. At this time, the determination unit uses the virtual camera (hereinafter referred to as “virtual camera”) set in the imaging condition information of the imaging unit to convert the place model in which the object model is arranged into 3D computer graphics. By performing the rendering process, a rendering image that is a virtual image corresponding to the captured image viewed from the imaging unit is generated. Then, by comparing the image feature of the image region corresponding to the object model (hereinafter referred to as “object image region”) in the generated rendered image with the image feature of the change region extracted from the captured image of the imaging unit. The movement of the object is determined. For example, when the position / size / shape of the object image area substantially matches the position / size / shape of the change area, it can be determined that the object is moving. As described above, the image monitoring apparatus of the present invention can determine the abnormal state of the object by considering the attribute information of the object changed by the monitored person and the image feature of the change area.

  According to the image monitoring apparatus according to the present invention, since the attribute information of the object can be dynamically changed according to the spatial positional relationship between the attention change region extracted from the captured image and the object, the monitor can It is possible to efficiently monitor abnormal conditions in the inside.

Block configuration diagram showing the overall configuration of the image monitoring device Diagram showing object table Diagram showing attribute table A diagram showing an example of a captured image and a reference image The figure explaining virtual space position calculation processing Flow chart showing processing procedure of overall operation by image monitoring apparatus A diagram representing a location model The flowchart which shows the specific process sequence of an attribute change process Flow chart showing specific processing procedure of determination processing Diagram showing examples of rendered image and difference image

  Hereinafter, an embodiment in which the inside of a building is used as a monitoring place, the captured image obtained by picking up the monitoring place with a monitoring camera is subjected to image processing, and an abnormal state in the monitoring place is monitored will be described in detail with reference to the accompanying drawings. . In the present embodiment, a change area is extracted from captured images acquired from a plurality of monitoring cameras, and the object is preferentially monitored according to a spatial positional relationship between the change area and an object grounded at a monitoring place. It is an object of the present invention to provide an image monitoring apparatus including a function for changing attribute information, which is information such as whether or not a power should be monitored, and determining an abnormal state of the object based on the attribute information.

(About the overall configuration of the image monitoring device)
As shown in FIG. 1, the image monitoring apparatus 1 according to the present embodiment includes a monitoring terminal 2 and an imaging apparatus 3.

  The imaging device 3 is a so-called surveillance camera including an imaging element such as a CCD element or a C-MOS element, an optical system component, and the like, and functions as an imaging unit in the present invention. The imaging device 3 is installed on the upper part or ceiling of the indoor wall, and is installed so as to take an image while looking down at the monitoring place. The imaging device 3 captures the monitoring location at predetermined time intervals and sequentially transmits the captured images to the monitoring terminal 2. The time interval at which the captured image is captured is, for example, 1/5 second. In the present embodiment, it is assumed that a plurality of imaging devices 3 are installed in the monitoring place.

  The monitoring terminal 2 is schematically configured to include a storage unit 21, a control unit 22, a communication unit 23, an output unit 24, and an input unit 25. The communication unit 23 is a communication interface such as a LAN or USB, and communicates with the imaging device 3. In the present embodiment, captured images transmitted from a plurality of imaging devices 3 are received via the communication unit 23. The input unit 25 is an information input device such as a keyboard, a mouse, a touch panel, or a portable storage medium reading device. The monitor or the like can use the input unit 25 to set information regarding various imaging conditions such as the installation position of each imaging device 3, for example.

  The storage unit 21 is an information storage device such as a ROM, RAM, or HDD. The storage unit 21 stores various programs and various data, and inputs and outputs such information to and from the image processing unit 22. The various data includes the location model 211, the imaging condition information 212, the object table 213, the attribute table 214, and other various information used for the processing of the image processing unit 22 (for example, the captured image for each frame acquired by the imaging device 3). , A reference image used for extracting a change area from the captured image, an extraction threshold value, and the like) are stored.

  The place model 211 is coordinate information representing a three-dimensional virtual space including three-dimensional shape data created by modeling an object such as a real world wall, floor, pillar, or fixture existing at the monitoring place. . That is, the place model 211 uses model information (object model) composed of three-dimensional shape data in a virtual space of an object existing at a monitoring place and position data (virtual space position) in the virtual space as individual components. Coordinate information that is configured. The three-dimensional shape data in the place model 211 may be created by three-dimensional CAD based on the shape information of the monitoring place, or the data obtained by capturing the three-dimensional shape of the monitoring place using a three-dimensional laser scanner or the like. Also good. The location model 211 created in this way is stored in the storage unit 21 by being set and registered from the input unit 25 by a supervisor or the like.

  The imaging condition information 212 includes installation condition information related to the installation position and optical axis (attitude) of the imaging device 3 at the current time, and field angle condition information related to the focal length, the number of pixels, the pixel size, and the lens distortion. Individually set for each. The imaging condition information 212 is set as a value associated with coordinate information in the virtual space of the place model. Here, the installation condition information regarding the installation position represents the inside of the monitoring place (real space) as a three-dimensional orthogonal coordinate system, and the coordinate value of the reference point whose coordinates are known in the orthogonal coordinate system of the real space is changed from the reference point to This is information expressed as coordinate data calculated using a known technique such as measuring and correcting the relative distance and direction. The installation condition information regarding the optical axis is information regarding the rotation angle of the optical axis of the imaging device 3 with respect to the coordinate axis, and can be obtained from the so-called pan angle and tilt angle of the imaging device 3. The imaging condition information 212 is stored in the storage unit 21 by being set and registered from the input unit 25 by a monitor or the like at the time of initial setting.

  As shown in FIG. 2, the object table 213 includes identification information of an object model that is a constituent element of the location model 211, and attribute information indicating information such as whether the object is in a monitoring state to be monitored with priority. Is a table that associates. In the present embodiment, as the attribute type of the attribute information, there are a watch-at-watch attribute for intensive monitoring when a person approaches the object, and a movement prohibition attribute for intensive monitoring when the object moves. It will be explained as being. In FIG. 2, when nothing is registered as attribute information, that is, an object that is not in a state to be monitored mainly is represented as “−”. The object table 213 is set and registered in advance by a supervisor or the like, and is updated by the attribute changing unit 223 of the control unit 22 described later.

  The attribute table 214 is information used by the attention changing area detecting unit 221 described later, and is a table that associates the image feature (attention feature amount) of the attention changing area with the attribute type, as shown in FIG. The attribute table 214 is set and registered in advance by a monitor or the like.

  The control unit 22 is configured by a computer including a CPU and the like, receives an input of a digitized image from the imaging device 3, and performs a captured image reading process as a series of processes shown in FIGS. 6, 8, and 9 to be described later. In order to execute extraction processing, attention change area detection processing, virtual space position calculation processing, attribute change processing, determination processing, and output processing, attention change area detection means 221, virtual space position calculation means 222, attribute change means 223, determination Means 224 are included.

  The attention change area detection unit 221 extracts an area having a luminance change from the captured image as a change area, and when the extracted change area has a predetermined image feature, the change area is the attention change area. Attention change area detection processing is detected. In the extraction process, a past captured image in which no moving object exists for each imaging device 3 is stored in the storage unit 21 in advance as a reference image, and a difference value between luminance values of the latest acquired captured image and the reference image is stored. An area that is equal to or greater than a predetermined extraction threshold stored in the unit 21 is extracted as a change area. At this time, a captured image of the background of the monitoring place, a captured image acquired in the past, or the like can be appropriately selected and used as the reference image. In the extraction process, as the extraction process, a process of labeling the extracted change area is performed. At this time, when attention is paid to an extraction pixel having a change area, the extraction pixels adjacent to the extraction pixel of interest are regarded as a group of extraction pixel areas. Assign a unique label. In the attention change area detection process, a process of detecting a change area having a predetermined image feature preset in the attribute table 214 of the storage unit 21 as the attention change area from the change areas extracted by the extraction process. Hereinafter, extraction processing and attention change area detection processing will be described with reference to the example of FIG. FIG. 4 a is a captured image 40 by an imaging device 3, and FIG. 4 b is a reference image 41 of the imaging device 3. That is, in the extraction process, the image areas indicated by reference numerals 30a and 30b in FIG. In the display change area setting process, the change area 30a that substantially matches the image feature set in the attribute table 214 is detected as the attention change area among the change areas 30a and 30b.

  The virtual space position calculation means 222 calculates the virtual space position of the change area from the imaging condition information 212 stored in the storage unit 21 and the position of the change area extracted by the attention change area detection means 221 on the captured image. Virtual space position calculation processing is performed. In the present embodiment, the virtual space position is coordinate information in the virtual space calculated so as to correspond to the coordinate information at the lower end center of the change area on the captured image, and the change area is grounded to the place model. Calculated as a value obtained assuming that Hereinafter, details of the virtual space position calculation processing will be described with reference to FIG.

  FIG. 5 illustrates an example of calculating the virtual space position of the change area based on the captured image of the imaging device 3 that captures the ground direction from above the monitoring location. In FIG. 5, reference numerals 211 a, 211 b, and 211 c represent a part of the location model 211, of which 211 a represents the floor surface of the building that is the monitoring location, 211 b represents the wall surface, and 211 c represents the monitoring location. It shall represent the safe placed. In calculating the virtual space position, the virtual space position calculation unit 222 first reads the installation position (X, Y, Z) of the imaging device 3 from the imaging condition information 212 of the storage unit 21, and a location corresponding to the position The optical center O on the model 211 (virtual space) is obtained. Further, the virtual space position calculation unit 222 calculates the light on the location / location model 211 of the imaging device 3 from the optical center O and the installation condition information regarding the optical axis (posture) read from the imaging condition information 212 of the storage unit 21. Find the axis. Further, the virtual space position calculation unit 222 reads out the imaging condition information 212 such as the focal length f, the actual size of the CCD pixel, the number of vertical and horizontal pixels of the image, and specifications relating to the lens distortion, from the storage unit 11. A projection plane abcd perpendicular to the optical axis is obtained on the location model 211. A quadrangular pyramid Oa′b′c′d ′ passing through the four vertices of the projection surface abcd from the optical center O is a solid that forms the field of view of the imaging device 3. In FIG. 5, it is assumed that an image area denoted by reference numeral 30 on the captured image projected on the projection surface abcd is a change area extracted by the attention change area detection unit 221.

  Next, the virtual space position calculation unit 222 obtains the foot position on the place model 211 (virtual space) of the change area object based on the position of the change area 30 on the captured image. A straight line connecting the lower end position 30a of the change area 30 and the optical center O is obtained, and an intersection point 31a with the place model 211 is obtained on an extension line of the straight line. In this embodiment, when there are a plurality of pixels located at the lower end of the change area, the pixel at the center position of the change area is set as the lower end position 30a. However, the present invention is not limited to this, and a predetermined right end (or left end) is used. The pixel may be the lower end position 30a. Since the intersection 31a corresponds to a position on the place model 211 (virtual space) corresponding to the position 30a on the captured image of the change area 30, the coordinates of the position in the virtual space are calculated as the virtual space position. In the example of FIG. 5, it can be seen that the change area object corresponding to the change area 30 is in contact with the place model of the safe 211c.

  The attribute changing unit 223 uses the virtual space position obtained by the virtual space position calculating unit 222 and the virtual space position of the object model in the place model 211 of the storage unit 21 to detect by the attention change region detecting unit 221. An attribute change process is performed to change the attribute information of the object model having a predetermined spatial positional relationship in the virtual space with the noted change area. For example, the spatial positional relationship is stored in advance in the storage unit 21 as positional relationship information, and “the virtual space position of the attention change region is within 1 m above the virtual space position of the object model” is set in the positional relationship information. The attribute information of the object model having such a spatial positional relationship with the attention changing area is changed. When changing the attribute information, the attribute changing unit 223 compares the image feature (attention feature amount) of the attention change area with the image feature of the attribute table 214, determines the attribute type of the attention change area, and determines the attribute Change attribute information using type. Details of the attribute change processing will be described later.

The determination unit 224 refers to the attribute information of the object table 213 stored in the storage unit 21, performs determination processing for determining the state of the object from the attribute information and the captured image, and outputs the determination result to the output unit 24. Process. Details of the determination process will be described later.
The output unit 24 is an interface for outputting processing results from the control unit 22 to various external devices. For example, when the output unit 24 is connected to a display output device such as a display and determines that the object at the monitoring place is in an abnormal state, the output unit 24 displays a warning message and a corresponding captured image of the imaging device 3 on the display output device. Output. The output unit 24 is connected to a sound output device such as a speaker or a buzzer, and outputs an abnormal signal that causes the sound output device to sound a warning sound in response to an instruction from the control unit 22. The monitor or the like can detect an abnormal state in the monitoring area by confirming the display output or the sound output from the output unit 24 as the warning output. The output unit 24 may include a communication interface that transmits an abnormal signal to an external security device (not shown). Thereby, when an abnormal state is detected by the processing of the control unit 22, the monitoring terminal 2 can notify a signal indicating the abnormal state to an external monitoring center or the like. In this case, the output unit 24 may be an interface device common to the communication unit 23.

  As described above, the image monitoring apparatus according to the present embodiment detects the marker set by the monitored person as the attention change area, and determines the attribute of the object according to the spatial positional relationship between the attention change area and the object model. Information can be changed. Therefore, for example, the person to be monitored can easily change the attribute information of the object simply by placing a marker on the object whose attribute information is to be changed. It is possible to perform the monitoring operation by limiting the monitoring target to the object to be operated. As a result, even in an environment where a large number of cameras are installed, monitoring can be performed without increasing the monitoring load on the observer.

(About processing executed by the control unit 22 of the monitoring terminal 2)
Hereinafter, an example of a flow of processing executed by the control unit 22 of the monitoring terminal 2 according to the image monitoring apparatus 1 of the present embodiment will be described with reference to FIG. In the present embodiment, description will be made assuming that four imaging devices are installed in a building that is a monitoring place.

  Prior to the operation, various initial settings such as setting of the imaging condition information 212, setting of the object table 213, setting of the attribute table 214, and registration of the location model 211 are performed by the monitor or the like using the input unit 25 of the monitoring terminal 2. (ST1). In this embodiment, since it is assumed that four imaging devices 3 are installed in a building that is a monitoring place, the imaging condition information 212 is registered by default for each imaging device 3. Further, the following processing will be described assuming that the three-dimensional shape data of FIG. In FIG. 7, reference numerals 34w, 34x, 34y, and 34z are object models that represent the imaging device 3, reference numeral 32 is an object model that represents a locker, and reference numeral 33 is an object model that represents a safe. In addition, the place model 211 is configured by various object models such as a floor surface, a wall, and a door.

  Next, the control unit 22 executes a captured image reading process for reading the captured image received from the imaging device 3 and stored in the storage unit 21 for each imaging device 3 (ST2). In the captured image reading process, the latest captured image (closest to the current time) is read for each imaging device 3.

  Next, an extraction process is executed by the attention changing area detecting means 221 of the control unit 22 (ST3). In the extraction process, each change area is extracted from the captured image of each imaging device 3 acquired in the captured image reading process. In this embodiment, the change area is extracted by taking the difference between the captured image acquired this time and the reference image. In the extraction process, a labeling process for labeling the extracted change area is also performed.

  Next, the attention change area detection unit 221 of the control unit 22 executes attention change area detection processing (ST4). In the attention change area detection process, a change area having a predetermined image feature preset in the attribute table 214 of the storage unit 21 is detected as the attention change area from the change areas extracted by the extraction process, and the change area Is stored in the storage unit 21. In this example, as illustrated in FIG. 4, it is assumed that the attention changing region 30 a is detected in the captured image 40 from the imaging device 3 corresponding to the object model 31 z in FIG. 7.

  Next, a virtual space position calculation process is executed by the virtual space position calculation means 222 of the control unit 22 (ST5). In the virtual space position calculation processing, the virtual space position of the change area is calculated from the position on the captured image of the change area extracted by the attention change area detection unit 221 and the coordinates of the virtual space position are used as the label of the attention change area. The process which matches and memorize | stores in the memory | storage part 21 is performed. Since the details of the virtual space position calculation process have been described above, the description thereof is omitted here.

  Next, the attribute changing unit 223 of the control unit 22 executes attribute changing processing (ST6). In the attribute change process, the virtual space position calculated in the virtual space position calculation process (ST5) of the attention change area 30a detected in the attention change area detection process (ST4) and the virtual space position of the object model are obtained. To change the attribute information of the object model. Details of the attribute change process will be described later.

  Next, determination processing is executed by the determination means 224 of the control unit 22 (ST7). In the determination process, the object state is determined based on the relationship between the attribute information and the change area extracted from the captured image with reference to the object table 213 and the attribute table 214 stored in the storage unit 21. Details of the determination process will be described later.

  Next, the determination unit 224 of the control unit 22 performs a process of outputting the determination result by the determination process of ST7 to the output unit 24 (ST8). Then, the image processing unit 22 repeatedly executes the processes of ST2 to ST8 until a termination operation is performed from the input unit 25 by a supervisor or the like.

(Details of attribute change processing)
Next, details of the attribute changing process executed by the attribute changing unit 223 of the control unit 22 in ST6 of the flowchart of FIG. 6 will be described with reference to FIG. Note that loop 1 in FIG. 8 means that each process of ST20 to ST22 is executed by the number of object models constituting the location model 211. In the description of this process, the selected object model refers to an object model that is a processing target in loop 1.

  In the attribute change process, first, it is determined whether or not the selected object model has a predetermined spatial positional relationship with the attention change area detected in ST4 (ST20). At this time, using the virtual space position of the attention change region calculated in ST5 and the virtual space position of the selected object model, determination is made based on whether or not the positional relationship information stored in advance in the storage unit 21 is met. . Note that the positional relationship information is a spatial positional relationship that is fixedly defined regardless of the type of the object model, for example, “the virtual space position of the target change region is within 1 m above the virtual space position of the object model”. Alternatively, it may be a spatial positional relationship defined for each type of object model such as “the virtual space position of the noticeable change region is within 1 m above the virtual space position of the object model representing the safe”. When it is determined in ST20 that the selected object model does not have a predetermined spatial positional relationship with the target change area (ST20-No), another object model that has not yet been processed in loop 1 is selected as the selected object model. Change the process from ST20.

  When it is determined in ST20 that the selected object model has a predetermined spatial positional relationship with the attention change region (ST20-Yes), the attribute changing unit 223 performs an attribute type determination process (ST21). In the attribute type determination process, the attribute changing unit 223 examines the image feature (attention feature amount) of the attention changing area having a predetermined spatial positional relationship with the selected object model, and refers to the attribute table 214 to determine the attribute type. To do. For example, in the example of FIG. 4, since the image feature of the attention changing area 30a corresponds to the priority monitoring attribute of the attribute table 214, the attribute of the safe object model 33 having a predetermined spatial positional relationship with the attention changing area 30a. The information is determined to be a priority monitoring attribute.

  Next, using the attribute type determined in the attribute type determination process, a process for changing the attribute information of the selected object model in the object table 213 is performed (ST22). For example, in the example of FIG. 4, the attribute information of the object model 33 of the safe having a predetermined spatial positional relationship with the attention changing area 30a is changed using the attribute type (priority monitoring attribute) determined in ST21. . When the process of ST22 is completed, another object model not yet processed in loop 1 is changed to the selected object model, and the process from ST20 is performed. When the processing for all the object models is completed in loop 1, the attribute changing unit 223 ends the attribute changing processing and advances the processing to ST7 in FIG.

(Details of judgment processing)
Next, details of the determination process executed by the determination unit 224 of the control unit 22 in ST7 of the flowchart of FIG. 6 will be described with reference to FIG. Note that the loop 2 in FIG. 9 means that each process of ST40 to ST45 is executed by the number of object models constituting the location model 211. In the description of this process, the selected object model refers to an object model that is a processing target in loop 2.

  In the determination process, first, the object table 213 is referred to and it is determined whether or not the selected object model is a priority monitoring attribute (ST40). When it is determined in ST40 that the selected object model is the priority monitoring attribute (ST40-Yes), it is determined whether or not another change area exists within a predetermined range of the selected object model (ST41). At this time, the determination unit 224 detects a predetermined range from the selected object model using the virtual space position of the selected object model and the range information preset in the storage unit 21 (for example, information such as a range within a radius of 5 m). An area is set on the location model 211. Then, the determination unit determines whether or not the change region 30 exists in the set detection area using the virtual space position of each change region calculated by the virtual space position calculation unit 222. When it is determined in ST41 that there is no other change region within the predetermined range of the selected object model (ST41-No), another object model not yet processed in loop 2 is changed to the selected object model. Then, the process from ST40 is performed.

  When it is determined in ST41 that another change region exists within the predetermined range of the selected object model (ST42-Yes), the determination unit 224 determines that the selected object model is in an abnormal state, and the selected object model The identification information and the information indicating the abnormal state are associated with each other and stored in the storage unit 21 (ST42).

  When it is determined in ST40 that the selected object model is not the priority monitoring attribute (ST40-No), it is determined whether or not the selected object model is a movement prohibition attribute (ST43). When it is determined in ST43 that the selected object model is not a movement prohibition attribute (ST43-No), another object model not yet processed in loop 2 is changed to the selected object model, and the process from ST40 is performed. I do.

  When it is determined in ST43 that the selected object model has a movement prohibition attribute (ST43-Yes), a movement determination process is performed to determine whether or not the object corresponding to the selected object model has moved (ST44). Details of the movement determination process will be described below.

  In the movement determination process, first, the imaging condition information 212 of the imaging device 3 that captures the selected object model is read from the storage unit 21, and the 3D computer graphics rendering process is performed using the location model 211 using the imaging condition information 212. By doing so, a rendering image that is a virtual image corresponding to the captured image 40 viewed from the imaging device 3 is generated. FIG. 10A shows a rendering generated using the imaging condition information 212 of the imaging device 3 (corresponding to the object model 31z in FIG. 7) that images the selected object model 33 when the selected object model is an object model 33 representing a safe. An example of the image 42 is shown. In FIG. 10a, reference numeral 42a denotes an image area (object image area) corresponding to the safe object model 33 which is the selected object model. FIG. 10 b shows a difference image 43 between the captured image and the reference image, which is used for extracting the change area from the captured image of the imaging apparatus 3 by the extraction process. That is, it means that the image area 30c in the difference image 43 is detected as a change area. As shown in FIG. 10b, when the object image area 42a and the change area 30c are compared, and the position / size / shape substantially matches the position / size / shape of the change area, it corresponds to the selected object model. It can be determined that the moving object is moving.

  In the present embodiment, the object image area 42a and the change area 30c are compared, and the object moves when the position / size / shape substantially matches the position / size / shape of the change area. However, the present invention is not limited to this, and the movement may be determined by comparing the shape of the object image region and the shape of the change region using a conventional technique such as edge matching.

  When the movement determination process ends in ST44, determination means 224 branches the process depending on whether or not the object corresponding to the selected object model has moved (ST45). When the object corresponding to the selected object model does not move (ST45-No), another object model not yet processed in loop 2 is changed to the selected object model, and the processing from ST40 is performed. On the other hand, when the object corresponding to the selected object model moves (ST45-Yes), it is determined as an abnormal state as described above (ST42). When the process for all the object models is completed in loop 2, the determination unit 224 ends the determination process and advances the process to ST8 in FIG.

  By the way, the present invention is not limited to the above-described embodiment, and may be implemented in various different embodiments within the scope of the technical idea described in the claims. Further, the effects described in the embodiments are not limited to this.

  In the above embodiment, in the attention change region detection process, the image feature of the selected change region is compared with the image feature (target feature amount) of the attention change region stored in the storage unit 21, and the image feature of the selection change region is the feature of interest. It is determined whether or not the selected change region is the attention change region by determining whether or not the amount matches. However, the present invention is not limited to this, and whether or not the change area of interest is included in the selected change area using a discriminator (so-called detector) that is a known prior art (see, for example, Japanese Patent Laid-Open No. 2008-26974) It may be determined whether or not the selected change area is the attention change area. Thereby, the attention change area included in the change area can also be detected.

  In the above embodiment, the imaging condition information 212 of the imaging device 3 is fixedly set in ST1 (initial setting) in FIG. However, the present invention is not limited to this, and the imaging condition information 212 is periodically received from the imaging device 3 via the communication unit 23 of the monitoring terminal 2, and the processes after ST2 are performed based on the received imaging condition information 212. Also good. Thereby, for example, even if the imaging condition information 212 is changed by panning, tilting, zooming, or the like with respect to the imaging apparatus 3, the monitor can follow the changed imaging condition information 212 while monitoring the state at the monitoring location. Can be monitored.

DESCRIPTION OF SYMBOLS 1 ... Image monitoring apparatus 2 ... Monitoring terminal 3 ... Imaging device 21 ... Memory | storage part 22 ... Control part 23 ... Communication part 24 ... Output part 25 ... Input part 211 ... Location model 212 ... Imaging condition information 213 ... Object table 214 ... Attribute table 221 ... Attention change area detection means 222 ... Virtual space position calculation means 223 ... Attribute change means 224 ... determining means 30 ... change area 40 ... captured image 41 ... reference image 42 ... rendered image 43 ... difference image

Claims (3)

An imaging unit that sequentially acquires captured images of the monitoring location;
Expressed by a location model that represents the monitoring location as a three-dimensional virtual space, a three-dimensional shape of an object that is a component of the location model and exists at the monitoring location, and a virtual space position that is a position in the virtual space An object table in which the object model is associated with the attribute information of the object model, an attribute table in which an image feature of the attention change area is associated with an attribute type of the attribute information, an installation position of the imaging unit, an optical axis And a storage unit that stores imaging condition information in which an angle of view is associated with the virtual space,
Attention change area detection means for extracting a change area from the captured image and detecting the change area as an attention change area when the change area has an image feature of the attention change area;
Virtual space position calculation means for obtaining a virtual space position of the target change area using the location model, the imaging condition information, and a position of the target change area on the captured image;
Wherein determining the attribute type of the noted change area image feature attribute table and by comparing the image feature of the target change area, by using the virtual spatial position of the object model and the virtual spatial position of the target change region An attribute changing means for changing the attribute information in the object table of the object model having a predetermined spatial positional relationship with the attention changing region in the virtual space according to the determined attribute type ;
An image monitoring apparatus comprising: a determination unit that determines a state of the object from the captured image using the object table and outputs a determination result. The virtual space position calculation means obtains the virtual space position of the change area using the place model, the imaging condition information, and the position of the change area on the captured image,
The determination means, the attribute information of the object model, and determining the positional relationship between the virtual space position of the object model and the virtual spatial position of the change area, the state of the object corresponding to said object model in accordance with the The image monitoring apparatus according to claim 1, wherein the determination result is output. The determination unit generates a rendering image corresponding to the field of view of the imaging unit, using the imaging condition information, for a place model where the object model is arranged, and an image region corresponding to the object model in the rendering image The movement of the object corresponding to the object model is determined by comparing the image feature with the image feature of the change area, and the attribute information of the object model corresponding to the object determined to have moved is prohibited from moving. when a mobile prohibit attribute, the image monitoring apparatus according to claim 1 or claim 2 outputs the determination result to the said object in an abnormal state.

Images