JP6448457B2 - Imaging direction variation detection apparatus and imaging direction variation detection method - Google Patents

Description

  This invention calculates position information of a person or an object from an image and compares the calculated position coordinates with a reference position coordinate serving as a reference, thereby detecting a change in the shooting direction of the image pickup apparatus. The present invention also relates to a method for detecting a change in photographing direction.

2. Description of the Related Art Conventionally, a technique for detecting that there has been a change in the shooting direction of an imaging apparatus is known.
For example, in Patent Document 1, a plurality of cameras having appearance characteristics different from those of other cameras are installed so as to photograph at least two other cameras, and camera images obtained by photographing the at least two other cameras. A technique for detecting a change in the orientation of the camera by calculating the position of the camera from the above and comparing the previously calculated position with the currently calculated position is disclosed.
Further, in Patent Document 2, a foreground image obtained by extracting only a difference from a background image from a captured image by a fixed point monitoring camera is generated, and a differential image of the continuous foreground image is divided into a plurality of blocks. Disclosed is a technique for detecting a change in the direction of a fixed-point monitoring camera depending on how many sudden change blocks there are in the difference image and the difference area occupancy ratio of the block is greater than or equal to a threshold value. ing.

JP 2011-076573 A JP 2014-026372 A

  However, in the technique disclosed in Patent Document 1, the camera must be installed so as to photograph at least two other cameras, so that the at least two other cameras cannot be photographed. If the direction of the camera is changed in a direction in which the shooting direction is completely different, there is a problem that a difference cannot be acquired and fluctuation detection cannot be normally performed.

  On the other hand, the technique disclosed in Patent Document 2 does not require an object that must be photographed, and uses a foreground image extracted from a captured image by a fixed-point monitoring camera. Therefore, even if the direction of the camera is changed to a direction in which the shooting direction is completely different, the difference can be acquired.

However, since the technique disclosed in Patent Document 2 does not consider how reliable the foreground image, that is, the object for detecting the difference in the difference detection, There is a problem that even if there is no change in the direction, it may be determined that there is a change.
For example, if a fallen object with a size that covers a range that is equal to or greater than the threshold of the range of the captured image suddenly falls, even if there is no change in the orientation of the fixed point monitoring camera, it may be determined that there has been a change. There is sex.

  The present invention has been made to solve the above-described problems, and more accurately considers the imaging direction of the imaging apparatus by taking into account the reliability of the object for detecting fluctuations in the imaging direction of the imaging apparatus. It is an object of the present invention to provide a photographing direction fluctuation detecting device capable of detecting the fluctuation of the image capturing direction.

  An imaging direction variation detection apparatus according to the present invention calculates position coordinates of an object on an image based on an image captured by an imaging apparatus, and the calculated position coordinates are compared with reference position coordinates to be compared with the position coordinates. The position coordinate calculation unit that performs grouping according to the reliability group that is set based on the reliability according to the movement possibility of the object, and the position coordinates calculated by the position coordinate calculation unit are grouped into a highly reliable group. The position coordinates are compared with the reference position coordinates in order from the divided position coordinates, and the position coordinates comparison section that determines whether the position coordinates match the reference position coordinates, and the position coordinates based on the determination result of the position coordinate comparison section. And a reference position coordinate, and when the calculated difference is equal to or greater than a threshold value, a fluctuation amount calculation unit that detects a fluctuation in the shooting direction of the imaging apparatus is provided.

  According to the imaging direction variation detection device and the imaging direction variation detection method according to the present invention, the reliability of the imaging direction of the imaging device can be more accurately considered by considering the reliability of the target for detecting the variation in the imaging direction of the imaging device. Variations can be detected.

It is a figure explaining the structure of the imaging | photography direction fluctuation | variation detection apparatus which concerns on Embodiment 1 of this invention. It is an internal block diagram of the direction fluctuation | variation determination part of Embodiment 1 of this invention. It is a figure explaining the example of calculation of the fluctuation | variation of the imaging | photography direction of the camera in Embodiment 1 of this invention. It is a figure explaining the example of calculation of the fluctuation | variation of the imaging | photography direction of the camera in Embodiment 1 of this invention. It is a flowchart explaining operation | movement of the imaging | photography direction fluctuation | variation detection apparatus which concerns on Embodiment 1 of this invention. 6 is a flowchart for explaining in detail an operation of a coordinate position calculation process in step ST503 of FIG. 5 by the position coordinate calculation unit in the first embodiment. In this Embodiment 1, it is a figure explaining an example of the reference position relevant information currently recorded on the recording medium. In Embodiment 1, it is a figure explaining the image of an example of grouping of the reliability group by a position coordinate calculation part. In Embodiment 1, it is a figure explaining an example of the content after the position coordinate comparison part rearranges the information regarding the object on the image which the camera temporarily preserve | saved by the position coordinate calculation part image | photographed. 6 is a flowchart for explaining in detail an operation of a reference position change presence / absence confirmation process in step ST505 of FIG. 5 by the reference position change presence / absence confirmation unit in the first embodiment. In Embodiment 1, it is a figure explaining an example of the content of the information regarding the object on the image which the camera after the rearrangement temporarily preserve | saved and image | photographed. 6 is a diagram for explaining an example of the content of reference coordinate related information recorded on a recording medium in Embodiment 1. FIG. 6 is a flowchart for explaining in detail the operation of a variation determination process in step ST508 of FIG. 5 by the variation amount calculation unit in the first embodiment. It is a figure explaining the hardware constitutions of the imaging | photography direction fluctuation | variation detection apparatus which concerns on Embodiment 1 of this invention. It is a block diagram of the imaging | photography direction fluctuation | variation detection apparatus which concerns on Embodiment 2 of this invention. It is an internal block diagram of the reference | standard position information setting control part of Embodiment 2 of this invention. 10 is a flowchart for explaining an operation of setting reference position related information and recording it on a recording medium in the second embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
Embodiment 1 FIG.
FIG. 1 is a diagram for explaining the configuration of the photographing direction variation detection apparatus 106 according to Embodiment 1 of the present invention.

As shown in FIG. 1, the shooting direction variation detection device 106 includes a video signal acquisition unit 101, a video signal processing unit 102, a direction variation determination unit 103, a recording medium 104, and an alarm unit 105.
The video signal acquisition unit 101 acquires video data captured by the camera 100 from an imaging device such as the camera 100.
The video signal processing unit 102 detects an object on an image captured by the camera 100 from the video data acquired by the video signal acquisition unit 101, and extracts an edge of the detected object.

  The orientation variation determination unit 103 is based on the edge feature amount extracted by the video signal processing unit 102, and the object on the image captured by the camera 100 from the edge acquired by the video signal acquisition unit 101 and extracted by the video signal processing unit 102. Are compared with the reference position coordinates recorded on the recording medium 104, the difference is calculated, and the presence or absence of a change in the photographing direction based on the difference is determined. In the first embodiment, the position coordinates are the coordinates of the midpoint of the edge of the object on the captured image of the camera 100.

The recording medium 104 stores reference position related information including reference position coordinates that are compared with the position coordinates of an object on an image captured by the camera 100 in advance. In the first embodiment, the reference position coordinate is, for example, a position coordinate on a captured image of an object existing within a shooting range captured by the camera 100 on a plane coordinate of a floor to be monitored. . The plane coordinates of the monitoring target floor are coordinate axes (common coordinate axes) common to a plurality of cameras that capture the same monitoring target, and the common coordinate axes are input by the user at the time of initial setting. When the user installs the camera 100, the user sets the camera position and shooting direction, and converts the coordinates on the image captured by the camera 100 of an object existing within the shooting range of the camera 100 into a common coordinate axis. Then, the reference position coordinates of the object are calculated.
Note that an object whose position coordinates can be used as reference position coordinates can be arbitrarily designated in advance by the user. For example, when the camera 100 is installed, the user can specify from objects existing on the image captured by the camera 100. Not limited to this, when the camera 100 is installed, an object that can be determined to be less moved or less moved from a captured image of the camera 100 for a predetermined period, such as being less than a set threshold, is automatically detected. You may make it specify.

In the first embodiment, as described above, when the user arbitrarily designates in advance or when the camera 100 is installed, there is no movement or a threshold set by the movement from the captured image of the camera 100 for a predetermined period. The position coordinates of the object that is designated by automatically detecting an object that can be determined to have little movement, such as less than, is converted into a common coordinate axis and recorded in advance on the recording medium 104 as a reference position coordinate.
For example, the position coordinates of the object designated by the user from the image captured when the camera 100 is installed on the image may be converted into a common coordinate axis and recorded in the recording medium 104 as the reference position coordinates. Of the images photographed by the camera 100, the position coordinates of the object designated by the user from the images photographed at the time arbitrarily acquired by the user are converted into common coordinate axes, and are recorded as reference position coordinates. 104 may be recorded. Further, for example, the position coordinates on the image of several frames of the object specified by the user are calculated from the images of several frames captured by the camera 100, the average value is converted into the common coordinate axis, and the recording medium is used as the reference position coordinates. 104 may be recorded.
In the first embodiment, the position information on the image captured in the normal state of the object present in the image captured in the normal state with the camera 100 set is converted into the common coordinate axis, and the reference position information Anything is acceptable.

The alarm unit 105 outputs an alarm when the direction variation determination unit 103 detects a variation in the shooting direction of the camera 100.
In the first embodiment, as shown in FIG. 1, the recording medium 104 is included in the shooting direction fluctuation detection device 106, but the recording medium 104 is not limited to this, and the recording medium 104 is used in the shooting direction fluctuation detection device 106. You may make it prepare outside.

FIG. 2 is an internal configuration diagram of the orientation variation determination unit 103 according to Embodiment 1 of the present invention.
As shown in FIG. 2, the direction variation determination unit 103 includes a position coordinate calculation unit 201, a position coordinate comparison unit 202, a reference position coordinate change presence / absence confirmation unit 203, and a variation amount calculation unit 204.
The position coordinate calculation unit 201 acquires the edges of all objects on the image captured by the camera 100 extracted by the video signal processing unit 102, and calculates the position coordinates of the object. Here, it is assumed that the position coordinate calculation unit 201 uses the coordinates of the midpoint of the edge of the object on the image captured by the camera 100 as the position coordinate of the object. In addition, the position coordinate calculation unit 201 calculates the coordinates converted into the common coordinate axis as the position coordinates of the object when calculating the position coordinates. In the following description, it is assumed that the position coordinates on the image of the object calculated by the position coordinate calculation unit 201 are converted to a common coordinate axis.
In addition, the position coordinate calculation unit 201 sets the reliability of the calculated position coordinates of the object on the basis of the reliability according to the object movement possibility with respect to the reference position coordinates to be compared with the position coordinates. Perform grouping according to group.

The position coordinate comparison unit 202 compares the position coordinates calculated by the position coordinate calculation unit 201 with the reference position coordinates stored in the recording medium 104 in order from the position coordinates grouped into a highly reliable group. It is determined whether the coordinates and the reference position coordinates match.
When the position coordinate comparison unit 202 determines that the position coordinates do not match the reference position coordinates, the reference position coordinate change presence / absence confirmation unit 203 determines the reliability of the reference position coordinates corresponding to the position coordinates determined not to match. In accordance with the group, it is determined whether or not the reference position coordinates to be compared are changed. If the reference position coordinate change presence / absence confirmation unit 203 determines that the reference position coordinates have been changed, the reference position coordinates recorded on the recording medium 104 are compared with the reference position coordinates recorded on the image taken by the camera 100 that has performed the comparison. Update with the contents of the position coordinates of the object.
The fluctuation amount calculation unit 204 determines the position coordinates according to the comparison result between the position coordinates and the reference position coordinates by the position coordinate comparison unit 202 and the determination result of whether or not the reference position coordinates are changed by the reference position coordinate change presence / absence confirmation unit 203. And the reference position coordinates are calculated, and when the calculated difference is equal to or greater than the threshold, a change in the shooting direction of the camera 100 is detected.

FIG. 3 and FIG. 4 show calculation examples of fluctuations in the shooting direction of the camera 100 according to Embodiment 1 of the present invention.
In FIG. 3, the case where the object 301 is changed to the position of the object 302, that is, the position coordinates are changed will be described.
An example of a camera image in which an image captured at the position of the object 301 is captured in the original capturing direction, and an example of a camera image in which the image captured at the position of the object 302 is captured as a result of a change in the capturing direction are used. That is, the position coordinates of the object 301 are converted into a common coordinate axis and recorded on the recording medium 104 as reference position coordinates.
In FIG. 4, a white circle 401 represents the reference position coordinates recorded on the recording medium 104 on the coordinates, and a black circle 402 represents the position coordinate calculation of the orientation variation determination unit 103 from the camera image obtained by photographing the object 302. The unit 201 calculates position coordinates and expresses them on the coordinates.
An arrow 403 represents the amount of change in the position coordinates from the position coordinates of the white circle 401 to the position coordinates of the black circle 402.

The operation will be described.
FIG. 5 is a flowchart for explaining the operation of the photographing direction fluctuation detecting apparatus 106 according to Embodiment 1 of the present invention.
Video signal acquisition section 101 acquires video data captured by camera 100 from camera 100 such as a camera (step ST501).
The video signal processing unit 102 detects an object on the image captured by the camera 100 from the video data acquired by the video signal acquisition unit 101 in step ST501, and extracts an edge of the detected object. (Step ST502). The edge is extracted as a feature amount, and a known image processing method can be used for extracting the edge. For example, the video signal processing unit 102 prepares a differential calculation method for extracting a sharp density value change portion of an edge as a function change, and eight types of masks corresponding to the direction of the edge, and is most similar to an image. An edge of a comparison target object is detected using a Prewitt filter using template matching for selecting a pattern.

In step ST502, if there are a plurality of objects on the image acquired from the video signal acquisition unit 101, the video signal processing unit 102 extracts all the edges of the plurality of objects.
Further, in step ST502, the information regarding the edge of the object on the image detected by the video signal processing unit 102 is temporarily stored in, for example, an internal memory or an empty space of the recording medium 104.

The position coordinate calculation unit 201 of the orientation variation determination unit 103 acquires the edge information of the object on the image extracted and temporarily stored by the video signal processing unit 102 in step ST502, and performs the position coordinate calculation process of the object ( Step ST503). Here, it is assumed that the position coordinate calculation unit 201 uses the coordinates of the midpoint of the edge as the position coordinates. In addition, the position coordinate calculation unit 201 calculates the coordinates converted into the common coordinate axis as the position coordinates of the object when calculating the position coordinates.
Further, at this time, the position coordinate calculation unit 201 sets the reliability group set based on the reliability according to the object movement possibility with respect to the reference position coordinate to be compared with the position coordinate of the target object on the image. Depending on the grouping.

Here, FIG. 6 is a flowchart illustrating in detail the operation of the coordinate position calculation process in step ST503 of FIG. 5 by the position coordinate calculation unit 201 in the first embodiment.
The position coordinate calculation unit 201 refers to the recording medium 104 and determines whether the reference position coordinates of the object on the image from which the video signal processing unit 102 extracted the edge in step ST502 is recorded on the recording medium 104 (step ST502). ST601). Specifically, the position coordinate calculation unit 201 performs matching between the edge feature amount extracted by the video signal processing unit 102 in step ST502 and the edge feature amount recorded in the recording medium 104 to match. It is determined whether there is an edge. If there is a matching edge, it is determined that the reference position coordinates corresponding to the object on the image are recorded on the recording medium 104, and the reference position coordinates are acquired.

Here, FIG. 7 is a diagram illustrating an example of the reference position related information recorded on the recording medium 104 in the first embodiment.
As illustrated in FIG. 7, information in which an object ID, an edge feature amount, a reference position coordinate, a reliability group, and a surrounding object ID are associated with each other is recorded on the recording medium 104. In the first embodiment, information recorded on the recording medium 104 as shown in FIG. 7 is used as reference position related information. The content shown in FIG. 7 is merely an example, and the reference position related information only needs to include at least the information shown in FIG.

  In step ST601, the position coordinate calculation unit 201 associates the edge feature amount extracted by the video signal processing unit 102 in step ST502 with the edge feature amount recorded in the recording medium 104 as shown in FIG. It is determined whether or not the feature values of the edges match.

When it is determined in step ST601 that the reference position coordinates are not recorded on the recording medium 104 (in the case of “NO” in step ST601), that is, the edge feature amount extracted by the video signal processing unit 102 in step ST502, When the feature value of the edge recorded on the recording medium 104 does not match, the position coordinate calculation unit 201 discards the object information on the image (step ST606), and proceeds to step ST605.
Note that an object such as a passerby that is considered not to exist on the image captured by the camera 100 in a normal state does not satisfy the condition that the position information is used as reference position information, and therefore information on an object that does not satisfy the condition is recorded. It is not recorded on the medium 104.

  When it is determined in step ST601 that the reference position coordinates are recorded on the recording medium 104 (in the case of “YES” in step ST601), that is, the edge feature amount extracted by the video signal processing unit 102 in step ST502, When the feature value of the edge recorded in the recording medium 104 matches, the position coordinate calculation unit 201 is recorded as a reliability group of the reference position coordinate of the feature value of the matched edge on the reference position related information. A reliability group is acquired (step ST602).

  For example, in step ST601, the position coordinate calculation unit 201 matches the edge feature amount of the object on the image with the edge feature amount of the object whose object ID is 1 as recorded in the recording medium 104 as shown in FIG. If it is determined that the reliability group 3 is recorded, the reliability group 3 recorded as the reliability group of the reference position coordinates of the feature amount of the edge of the object having the object ID of 1 is acquired.

The reliability group is based on the possibility of movement of the object, and the reference position coordinate is reliable as a target to be compared with the position coordinate of the object on the image in order to detect the amount of change in the shooting direction of the camera 100. The degree of existence is grouped and how much the position is difficult to change, i.e., how reliable it is as a comparison object for difference calculation to determine whether the shooting direction has changed These are grouped according to their reliability.
Note that the reliability group is set in advance by the user.

  If the reliability of the reference position coordinates is high, it is considered that there is a high possibility that the object will not move.The higher the reliability group set for the reference position coordinates is, the more the reference position coordinates are It is considered that there is a high possibility of being reliable as the position coordinates used when calculating the difference for determining whether there is any fluctuation. Details of the determination of the presence / absence of changes in the shooting direction will be described later.

For example, since the feature amount of an edge such as a column or wall does not change, the calculated coordinate position is considered to be highly reliable, and therefore, the reliability group is set to a higher group.
Here, the reliability group is grouped into three groups, and the reliability group 1 having the highest reliability is set as the reference position coordinates of an object that is difficult to move, such as a column or a wall. Further, the reliability group 2 having the next highest reliability is set for the reference position coordinates of an object that moves little, such as a shelf or a desk. Further, the reliability group 3 having the lowest reliability is set for the reference position coordinates of the portable object.

FIG. 8 is a diagram for explaining an image of an example of grouping of reliability groups by the position coordinate calculation unit 201 in the first embodiment.
FIG. 8A shows an example of an image captured by the camera 100. From the image, the video signal processing unit 102 uses a wall edge 701, a shelf edge 702, and the like as edges of a comparison target object. An example in which a ball edge 703 and a person edge 704 are extracted is shown.
FIG. 8B shows the comparison target object having each edge shown in FIG. 8A and the reference position related information recorded on the recording medium 104, and sets a reliability group. Shows an example of grouping.
As shown in FIG. 8B, the wall having the edge 701 is grouped in the first reliability group, and the wall having the edge 701 and the shelf having the edge 702 are grouped in the second or higher reliability group. It is divided. That is, the reliability group of the reference position coordinates corresponding to the shelf is set to the second place.

Further, the wall having the edge 701, the shelf having the edge 702, and the ball having the edge 703 are grouped in the third or higher reliability group. That is, the reliability group of the reference position coordinates corresponding to the ball is set at the third position.
In FIG. 8B, a reliability group corresponding to the person having the edge 704 in FIG. 8A is not set. This is because a person is considered not to exist on an image captured by the camera 100 in a normal state, and the reference position coordinates of such an object do not satisfy the condition, so that information is not recorded on the recording medium 104. This is because the information about the person has been discarded (see step ST606).

  Here, the reliability group is recorded in advance on the recording medium 104. However, the present invention is not limited to this, and the position coordinate calculation unit 201 stores a grouping rule for the reliability group in advance. A reliability group may be set according to the rule from the feature amount of the edge.

  In addition, although the reliability group is described as being grouped into three groups here, the present invention is not limited to this, and the group may be further finely grouped, and the number of groups can be set as appropriate.

The position coordinate calculation unit 201 calculates the position coordinates of the object on the image captured by the camera 100 (step ST603). Here, it is assumed that the position coordinate calculation unit 201 uses the coordinates of the midpoint of the edge as the position coordinates of the object on the image captured by the camera 100. In addition, the position coordinate calculation unit 201 calculates the coordinates converted into the common coordinate axis as the position coordinates of the object when calculating the position coordinates.
As described above, the video signal processing unit 102 detects an object on the image captured by the camera 100 from the video data acquired by the video signal acquisition unit 101, and extracts an edge of the detected object. Since the coordinate calculation unit 201 calculates the position coordinates of the object from the edge information, the processing load can be reduced.

  The position coordinate calculation unit 201 calculates information about the edge of the object on the image captured by the camera 100, the reference position coordinates acquired in step ST601, the reliability group of the reference position coordinates acquired in step ST602, and the calculation in step ST603. The information on the position coordinates is associated and temporarily stored in, for example, an internal memory or an empty space on the recording medium 104 (step ST604). That is, the position coordinate calculation unit 201 performs grouping on the position coordinates of the object on the image captured by the camera 100 based on the reliability group of the reference position coordinates.

The position coordinate calculation unit 201 determines whether or not the matching of the reference position related information recorded on the recording medium 104 has been completed for all the edges of the object on the image extracted by the video signal processing unit 102 in step ST502. (Step ST605).
If it is determined in step ST605 that all the matching has been completed (in the case of “YES” in step ST605), the processing in FIG. 6 is terminated.

  If it is determined in step ST605 that all matching has not been completed (in the case of “NO” in step ST605), the process returns to step ST601 and matching with reference position related information still recorded on the recording medium 104 is performed. The object edge on the image that has not been finished is read, and the subsequent processing is repeated. In this way, the position coordinate calculation unit 201 matches all the edges extracted by the video signal processing unit 120 in step ST502 with the reference position related information recorded on the recording medium 104.

  In the first embodiment, it is assumed that the reference position coordinates can be acquired with reference to another recording medium 104 acquired and recorded by another camera 100. Thereby, it is possible to reduce the occurrence probability of the feature amount of the edge whose pattern does not match.

Here, if it is determined that the reference position coordinates are not recorded on the recording medium 104, the information on the comparison target object is discarded (see step ST606). When it is determined that the image is not recorded on the recording medium 104, that is, when the feature value of the edge of the object on the image captured by the camera 100 does not match the feature value of the edge recorded on the recording medium 104, for example, Alternatively, it may be determined that the camera has been shielded or turned in the sky or the like, and an alarm may be generated. The alarm output operation (step ST510 in FIG. 5) will be described later.
Note that the processing order of step ST602 and step ST603 in FIG. 6 may be interchanged.

Returning to the flowchart of FIG.
In step ST503, when the position coordinate calculation unit 201 of the direction variation determination unit 103 calculates the position coordinates of the object on the image captured by the camera 100, the position coordinate comparison unit 202 of the direction variation determination unit 103 includes the position coordinate calculation unit. 201 rearranges the information temporarily stored in step ST604 of FIG. 6 in the order of the reliability of the reliability group of the reference position coordinates (step ST504). That is, here, the information on the comparison target object is rearranged so that the reliability group 1 is followed by the reliability group 2 and the reliability group 3 in this order. When there are a plurality of comparison target objects in which the same reliability group is set, the order of the comparison target objects in the same trust group is arbitrary.
FIG. 9 shows an example of the content after the position coordinate comparison unit 202 rearranges the information on the object on the image captured by the camera 100 temporarily stored by the position coordinate calculation unit 201 in step ST504.

The position coordinate comparison unit 202 sequentially refers to the information on the objects on the images taken by the camera 100 rearranged in step ST504, and determines whether the position coordinates match the reference position coordinates corresponding to the position coordinates. Determination is made (step ST505).
That is, the position coordinate comparison unit 202 performs comparison with the reference position coordinates from the position coordinates of objects that are grouped into higher reliability groups with high reliability.

  If it is determined in step ST505 that the position coordinates match the reference position coordinates (in the case of “YES” in step ST505), the position coordinate comparison unit 202 is temporarily stored by the position coordinate calculation unit 201 in step ST503, and the step It is determined whether or not the matching of the position coordinates of all the objects included in the information related to the object on the image captured by the camera 100 that has been rearranged in ST504 is completed with the reference position coordinates (step ST511). .

  In step ST511, when it is determined that the matching of the position coordinates and the reference position coordinates has not been completed for all the comparison target objects (in the case of “NO” in step ST511), the process returns to step ST505, and the position coordinates and the reference still Information on the next comparison target object that has not been matched with the position coordinates is read, and the subsequent processing is performed.

  If it is determined in step ST511 that the matching of the position coordinates of all the objects with the reference position coordinates has been completed (in the case of “YES” in step ST511), it is determined that there is no change in the shooting direction of the camera 100, and the process finish.

  When it is determined in step ST505 that the position coordinates of the object on the image captured by the camera 100 do not match the reference position coordinates (in the case of “NO” in step ST505), whether or not the reference position coordinates of the orientation variation determination unit 103 are changed The confirmation unit 203 performs reference position change presence / absence confirmation processing for determining whether or not there is a change in the compared reference position coordinates according to the reliability group of the reference position coordinates corresponding to the position coordinates determined not to match. It performs (step ST506).

Here, FIG. 10 is a flowchart for explaining in detail the operation of the reference position change presence / absence confirmation processing in step ST506 of FIG. 5 by the reference position coordinate change presence / absence confirmation unit 203 in the first embodiment.
The reference position coordinate change presence / absence confirmation unit 203 is a reference position coordinate that is temporarily stored in step ST503 by the position coordinate calculation unit 201 and is determined not to match the position coordinate in step ST505 from the information rearranged in step ST504. And whether or not the acquired reliability group is the top two groups, that is, whether or not it is the reliability group 1 or the reliability group 2 (step ST1001).

  When it is determined in step ST1001 that the reliability group of the reference position coordinates determined not to coincide with the position coordinates in step ST505 is not the upper two groups (in the case of “NO” in step ST1001), whether or not the reference position coordinates are changed is confirmed. Unit 203 determines whether or not there are 1/3 or more objects whose position coordinates do not match the reference position coordinates among objects around the object having the position coordinates (step ST1002). Specifically, the reference position coordinate change presence / absence confirmation unit 203 refers to the reference position related information recorded on the recording medium 104 from the edge feature value of the corresponding object, and is associated with the matching edge feature value. Obtain a peripheral object ID. Then, the edge feature amount of the peripheral object is acquired based on the acquired peripheral object ID. Thereafter, using the acquired edge feature amount of the peripheral object as a key, the information about the object on the image captured by the camera 100 that has been temporarily stored is matched, and the positional information associated with the matched edge feature amount It is determined whether or not the reference position information matches. And it is determined whether the object which does not correspond is 1/3 or more.

Here, the processing of step ST1001 to step ST1002 will be described with a specific example.
In the following example, for example, the content of the information on the object on the image captured by the rearranged camera 100 that is temporarily stored is FIG. 11, and the reference coordinate related information recorded on the recording medium 104 is shown in FIG. Assuming that the contents are FIG. 12, in step ST505 in FIG. 5, the position coordinate comparison unit 202 determines that the position coordinates of the object having the edge feature amount ff in FIG. 11 do not match the reference position coordinates.

  Since the reliability group of the reference position coordinates to be compared with the position coordinates of the object having the edge feature amount ff is 3, in step ST1001, the reference position coordinate change presence / absence confirmation unit 203 determines that the acquired reliability groups are the upper two groups. (NO in step ST1001).

Next, the reference position coordinate change presence / absence confirmation unit 203 refers to the recording medium 104 and acquires the peripheral object IDs 7, 8, and 9 associated with the edge feature amount ff.
Then, the reference position coordinate change presence / absence confirmation unit 203 acquires the edge feature amounts gg, hh, and ii of the peripheral object IDs 7, 8, and 9, respectively.
After that, the reference position coordinate change presence / absence confirmation unit 203 uses the edge feature values gg, hh, and ii as keys to perform matching with information on the object on the image captured by the camera 100 in FIG. , Position coordinates (x ₇ , y ₇ ), reference position coordinates (x ₇ , y ₇ ), edge feature quantity hh, position coordinates (x ₈ , y ₈ ), reference position coordinates (x ₈ , y ₈ ), edge For the feature quantity ii, the position coordinates (x ₉ , y ₉ ) and the reference position coordinates (x ₉ , y ₉ ) are acquired, and it is determined whether or not an object whose position coordinates do not match the reference position coordinates is 1/3 or more. . Here, since the position coordinates corresponding to all the edge feature values gg to ii match the reference position coordinates, the reference position coordinate change presence / absence confirmation unit 203 determines that an object whose position coordinates do not match the reference position coordinates is the same. It is determined that it is not 1/3 or more (“NO” in step ST1002).

  When the edge feature amount specified from the peripheral object ID is matched with the information in FIG. 11 with reference to the recording medium 104, if there is no matching edge feature amount, the position coordinate and the reference position coordinate are set. May not match or may not be counted.

Returning to the flowchart of FIG.
If it is determined in step ST1002 that the object whose position coordinates do not match the reference position coordinates is not 1/3 or more (in the case of “NO” in step ST1002), the reference position coordinate change presence / absence confirmation unit 203 determines in step ST505. The reference position coordinates determined not to match the position information have changed, that is, the reference position coordinates currently recorded on the recording medium 104 are coordinates for comparing with the position coordinates of the object on the image captured by the camera 100. (Step ST1003), the position coordinate calculation unit 201 uses the position coordinate calculation unit 201 to obtain the reference position coordinate information associated with the edge feature amount of the corresponding object recorded on the recording medium 104 in Step ST503. The calculated position coordinate information is updated (step ST1004). Specifically, in the example described above, the reference position coordinates (x ₆ , y ₆ ) associated with the edge feature value ff and currently recorded on the recording medium 104 are the position coordinates (x ₆₆ , Update to y ₆₆ ).

Then, reference position coordinate change presence / absence confirmation unit 203 sets 1 to the recording medium update flag (step ST1005). Note that the initial value of the recording medium update flag is 0.
In this way, the reference position coordinates for which a reliability group that is considered to have low reliability, such as an object that can be carried, is changed as the object moves and cannot be used as a comparison target. If the reference position coordinates are determined to be changed, the reference position coordinates can be updated to update the reference position coordinate information to the latest correct information.
Then, the process of FIG. 10 is complete | finished.

On the other hand, when it is determined in step ST1001 that the reliability group of the reference position coordinates determined not to coincide with the position coordinates in step ST505 is that the upper two groups are the upper two groups (in the case of “YES” in step ST1001). Alternatively, when it is determined in step ST1002 that the object whose position coordinates do not match the reference position coordinates is 1/3 or more (in the case of “YES” in step ST1002), the reference position coordinate change presence / absence confirmation unit 203 There is no change in coordinates, and in step ST505, the position coordinate comparison unit 202 determines that the determination content that the position coordinate and the reference position coordinate do not match is valid, and in step ST503, the position coordinate calculation unit 201 temporarily stores it. Camera after rearrangement in step ST504 00 is the information about the object on the photographed image, to impart object position moving flag 1 (step ST 1006). Note that the initial value of the object position movement flag is 0.
Then, the process of FIG. 10 is complete | finished.
Thus, if the comparison is made with the position coordinates from the upper group of the reliability group of the reference position coordinates and it is determined that the reference position coordinates of the upper group of the reliability group do not match, the determination content that does not match is It is determined that it is valid, and the other position coordinates are not compared with the reference position coordinates. By doing so, it is possible to reduce the load of the determination process and increase the efficiency of the process.

  In the first embodiment, in step ST1001, the reference position coordinate change presence / absence confirmation unit 203 determines whether or not the reliability group of the reference position coordinates that did not match the position coordinates is the upper two groups. However, the present invention is not limited to this, and only the top one group may be used. Up to which reliability group can be used as a comparison target unconditionally, that is, a reference position that does not need to consider the possibility of moving. Whether to use coordinates can be set as appropriate.

  In the first embodiment, in step ST1002, the reference position coordinate change presence / absence confirmation unit 203 has 1/3 or more objects whose position coordinates and reference position coordinates do not match among objects around the comparison target object. However, the present invention is not limited to this, and may be ½ or more, or may be ¼ or more, and the determination criteria can be set as appropriate.

Returning to the flowchart of FIG.
In step ST506, when the reference position coordinate change presence / absence confirmation unit 203 performs the reference position change presence / absence confirmation process, the fluctuation amount calculation unit 204 of the orientation variation determination unit 103 causes the reference position coordinate change presence / absence confirmation unit 203 to perform the recording medium update flag. It is determined whether or not 1 is set (step ST507).

If 1 is set in the recording medium update flag in step ST507 (in the case of “YES” in step ST507), the process returns to step ST505, and the position coordinate comparison unit 202 determines that the camera 100 rearranged in step ST504 With reference to the information regarding the object on the photographed image, it is determined whether or not the position coordinate of the next object matches the reference position coordinate corresponding to the position coordinate.
The fact that the recording medium update flag is set to 1 means that the reference position coordinates have moved and the reference position coordinates recorded on the recording medium 104 are no longer usable as a comparison target. In ST505, the correct comparison between the position coordinates and the reference position coordinates is not performed. Therefore, in that case, the position coordinates and the reference position coordinates are compared again for another object.
By doing so, it is possible to prevent the determination of the change in the shooting direction based on an erroneous determination result by comparison with an incorrect reference position coordinate. Note that the change in the shooting direction will be described later.
In step ST507, when returning to step ST505, the recording medium update flag is initialized to 0.

  In step ST507, when 1 is not set in the recording medium update flag (in the case of “NO” in step ST507), the fluctuation amount calculation unit 204 has changed in the shooting direction of the camera 100 or has changed. Performs a variation determination process for calculating the variation amount (step ST508).

Here, FIG. 13 is a flowchart for explaining in detail the operation of the variation determination process in step ST508 of FIG. 5 by the variation amount calculation unit 204 in the first embodiment.
The fluctuation amount calculation unit 204 determines whether or not 1 is set in the object position movement flag in the reference position change presence / absence confirmation processing in step ST506 (step ST1301).
When it is determined in step ST1301 that the object position movement flag is set to 1 (in the case of “YES” in step ST1301), the fluctuation amount calculation unit 204 is on the temporarily captured image captured by the camera 100. The difference between the position coordinates and the reference position coordinates is calculated with reference to the information related to the object (step ST1302).

For example, assuming that information with contents as shown in FIG. 9 is temporarily stored and the variation determination is performed on the position coordinates of the edge feature amount ee, the variation amount calculation unit 204 includes the object position in the edge feature amount ee. Since the movement flag 1 is set, it is determined in step ST1301 that it is the object position movement flag 1, and the difference between the position coordinates (x ₅₅ , y ₅₅ ) and the reference position coordinates (x ₅ , y ₅ ) is calculated. Note that the difference between the position coordinates and the reference position coordinates may be obtained by, for example, the distance between two points.

  The fluctuation amount calculation unit 204 determines whether or not the difference between the position coordinates calculated in step ST1302 and the reference position coordinates is equal to or greater than a preset threshold value (step ST1303). The threshold value determined here can be set as appropriate.

In step ST1303, when it is determined that the difference between the position coordinates and the reference position coordinates is greater than or equal to a preset threshold value (in the case of “YES” in step ST1303), the fluctuation amount calculation unit 204 determines the shooting direction of the camera 100. As a result, the shooting direction variation flag 1 is set and temporarily stored in the internal memory or the free space of the recording medium 104 (step ST1304). Note that the initial value of the shooting direction variation flag is 0.
At this time, the fluctuation amount calculation unit 204 uses the difference between the position coordinates calculated in step ST1302 and the reference position coordinates as a fluctuation amount, and temporarily stores the fluctuation amount, the position coordinates, and the reference position coordinates together.

The fluctuation amount calculation unit 204 calculates the current shooting direction of the camera 100, that is, the direction of the camera 100 (step ST1305). Specifically, the fluctuation amount calculation unit 204 determines the current camera direction of the camera 100 from the position coordinates temporarily stored in step ST1304, the reference position coordinates, the fluctuation amount, and the camera direction when the camera 100 is installed. calculate. At this time, the fluctuation amount calculation unit 204 stores the calculated information on the camera orientation of the current camera 100 in association with the information stored in step ST1304.
It is assumed that the camera direction at the time of installation of the camera 100 is stored in the photographing direction variation detection device 106 at the time of installation.
Then, the process of FIG. 13 ends.

  When it is determined in step ST1301 that the object position movement flag is not set to 1 (in the case of “NO” in step ST1301), or in step ST1303, the difference between the position coordinates and the reference position coordinates is set in advance. If it is determined that the value is not equal to or greater than the threshold value (in the case of “NO” in step ST1303), it is determined that there is no change in the shooting direction of the camera 100, and the processing in FIG.

Returning to the flowchart of FIG.
In step ST508, the alarm unit 105 determines whether or not 1 is set in the shooting direction variation flag by the variation amount calculation unit 204 (step ST509).
In step ST509, when 1 is not set in the shooting direction variation flag (in the case of “NO” in step ST509), it is determined that there is no variation in the shooting direction of the camera 100, and the process ends.
In step ST509, when the shooting direction variation flag is set to 1 (in the case of “YES” in step ST509), the alarm unit 105 outputs a warning that there is a variation in the shooting direction of the camera 100 (step ST509). ST510).
For example, the alarm unit 105 may output an alarm sound from a sound output device such as a speaker (not shown in FIG. 1) or a display device such as a display (FIG. 1). Warning information indicating that there has been a change in the shooting direction may be displayed.
In addition to outputting an alarm, information on the current shooting direction of the camera 100 calculated by the variation calculation unit 204 in step ST1305, that is, information on the orientation of the camera 100 is transmitted to, for example, a turning camera (not shown). Then, in a turning camera or the like, it is determined whether there is a blind spot in the shooting direction of the camera 100. If there is a blind spot, control for correcting the camera direction of the camera 100 may be performed. Alternatively, during the maintenance work of the camera 100, the operator can use the current shooting direction of the camera 100 calculated by the variation calculation unit 204 in step ST1305 as an index of how much the orientation of the camera 100 should be adjusted. May be notified.

FIG. 14 is a diagram for explaining the hardware configuration of the photographing direction variation detection apparatus 106 according to Embodiment 1 of the present invention.
In Embodiment 1 of the present invention, the recording medium 104 uses an HDD 1402. This is merely an example, and the recording medium 104 may be configured by a DVD, a memory 1403, or the like.
The video signal acquisition unit 101 uses a communication I / F device 1406.
The video signal processing unit 102, the position coordinate calculation unit 201, the position coordinate comparison unit 202, the reference position coordinate change presence / absence confirmation unit 203, and the fluctuation amount calculation unit 204 execute programs stored in the HDD 1402, the memory 1403, and the like. It is realized by a processing circuit such as a CPU 1401 and a system LSI.
In addition, a plurality of processing circuits may cooperate to execute the above function.
The alarm unit 105 uses a speaker 1404 or a display 1405. This is an example, and the alarm unit 105 may be configured by other hardware.

  As described above, according to the first embodiment, the position coordinates of the object on the image photographed by the camera 100 are calculated, and the change in the photographing direction of the camera 100 based on the difference between the position coordinates and the reference position coordinates. When determining the reference position, the position coordinates are grouped in consideration of the reliability of the reference position coordinates to be compared with the position coordinates, and the reference position is determined based on the position information grouped into a group with high reliability of the reference position coordinates. Since the comparison with the coordinates is performed, a change in the photographing direction of the camera 100 can be detected more accurately.

Embodiment 2. FIG.
In the first embodiment, based on the edges of all the objects extracted from the video captured by the camera 100, the corresponding reference position coordinates are acquired from the recording medium 104, and the group is based on the reliability group of the reference position coordinates. The reference position coordinates are compared from the position coordinates grouped into highly reliable groups, but the reference position coordinates recorded in the recording medium 104 are changed by object rearrangement or the like. When a large number of reference position coordinates are changed, there is a possibility that the update of the recording medium cannot catch up and the change in the shooting direction of the camera 100 cannot be accurately detected.
Therefore, in the second embodiment, an embodiment in which the reference position related information including the reference position coordinates recorded on the recording medium 104 is reflected in real time will be described.

FIG. 15 is a configuration diagram of the photographing direction variation detection apparatus 106 according to Embodiment 2 of the present invention.
In FIG. 15, the same components as those described in Embodiment 1 with reference to FIG.
As shown in FIG. 15, the photographing direction variation detection device 106 according to the second embodiment of the present invention is different from the photographing direction variation detection device 106 in FIG. 1 in that the reference position coordinate information acquisition unit 107 and the reference position information setting. The only difference is that the control unit 108 is further provided.

  The reference position coordinate information acquisition unit 107 constantly acquires images taken by the camera 901 and information on the installation position and shooting direction of the camera 901. Note that the camera 901 is an image pickup apparatus different from the camera 100 that attempts to detect a change in the shooting direction, and takes at least a range in which the camera 100 is shooting. Note that the camera 901 can increase the feature amount of the edge recorded on the recording medium 104 by using an imaging device capable of detecting an object in a wide range, for example, a turning camera or an omnidirectional camera.

  The reference position information setting control unit 108, based on the captured image of the camera 901 acquired by the reference position coordinate information acquisition unit 107, and information on the installation position and shooting direction of the camera 901, and the reference position coordinate Is set and recorded on the recording medium 104.

FIG. 16 is an internal configuration diagram of the reference position information setting control unit 108 according to the second embodiment of the present invention.
As shown in FIG. 16, the reference position information setting control unit 108 includes an edge extraction unit 1601, a reference position coordinate determination unit 1603, a reliability group setting unit 1604, a peripheral object ID setting unit 1605, and a reference position information setting. Part 1606 is provided.
The edge extraction unit 1601 acquires all the objects on the captured image of the camera 901 acquired by the reference position coordinate information acquisition unit 107, and extracts the edges of all the acquired objects.
The reference position coordinate determination unit 1603 sets the reference position coordinates of each object on the captured image based on the edge information of all objects on the captured image of the camera 901 extracted by the edge extraction unit 1601.
A reliability group setting unit 1604 sets a reliability group of reference position coordinates corresponding to each object.
The peripheral object ID setting unit 1605 sets the object ID of an object around each object as the peripheral object ID.
The reference position information setting unit 1606 sets reference position related information to be recorded on the recording medium 104, and records the set reference position related information on the recording medium 104.

The operation will be described.
Hereinafter, only the operation different from the operation described in the first embodiment will be described with respect to the operation of the imaging direction variation detection device 106 according to the second embodiment, and the operation already described in the first embodiment will be described. A duplicate description is omitted.
Regarding the operation of the photographing direction variation detection apparatus 106 according to the second embodiment, first, the position coordinate calculation method by the position coordinate calculation unit 201 of the direction variation determination unit 103 is different from that of the first embodiment. In the first embodiment, the common coordinate axis is the plane coordinate of the floor to be monitored input by the user. However, in the second embodiment, the coordinate axis on the captured image of the reference camera 901 is the common coordinate axis. . That is, the position coordinate calculation unit 201 calculates the coordinates converted into the coordinate axes on the captured image of the camera 901 as the position coordinates of the object when calculating the position coordinates.
In addition, in the shooting direction variation detection device 106 according to the second embodiment, an operation for calculating the reference position coordinates and recording it on the recording medium 104 is added to the operation of the shooting direction variation detection device 106 described in the first embodiment. Is done.
The photographing direction variation detection apparatus 106 according to the second embodiment of the present invention is described in the first embodiment with reference to the recording medium 104 in which the reference position related information including the reference position coordinates is set by the operation described below. As described with reference to FIG. 5, a change in the shooting direction of the camera 100 is detected.

FIG. 17 is a flowchart for explaining the operation of setting reference position related information and recording it on the recording medium 104 in the second embodiment.
The reference position coordinate information acquisition unit 107 constantly acquires an image captured by the camera 901 and information on the installation position and shooting direction of the camera 901 (step ST1701).
The edge extraction unit 1601 of the reference position information setting control unit 108 acquires all objects on the captured image of the camera 901 acquired by the reference position coordinate information acquisition unit 107, and extracts the edges of all the acquired objects. (Step ST1702). Note that the edge is detected as a feature amount, and a known image processing method can be used to detect the edge. For example, the edge extraction unit 1601 prepares a differential calculation method for extracting a sharp density value change portion of an edge as a function change, and eight types of masks corresponding to the direction of the edge. The edge of the object is detected using a Prewitt filter using template matching. At this time, the edge extraction unit 1601 assigns an object ID for each extracted edge. The object ID to be given may be set as appropriate, such as a serial number from 1, for example, and the object ID only needs to be not duplicated.

  The reference position coordinate determination unit 1603 of the reference position information setting control unit 108 sets the reference position coordinates of each object on the captured image based on the edge information extracted by the edge extraction unit 1601 in step ST1702 (step ST1703). Specifically, the reference position coordinate determination unit 1603 sets the coordinates of the midpoint of the edge of the object extracted by the edge extraction unit 1601 as the reference position coordinates.

  The reliability group setting unit 1604 of the reference position information setting control unit 108 sets a reliability group for each reference position coordinate (step ST1704). Specifically, for example, the reliability group setting unit 1604 performs matching with data in which the edge feature value and the reliability group are associated in advance using the edge feature value extracted in step ST1702 as a key, The reliability group associated with the feature value of the matched edge may be set as the reliability group of the corresponding reference position coordinate.

  The peripheral object ID setting unit 1605 of the reference position information setting control unit 108 sets the object ID of the object around each object as the peripheral object ID (step ST1705). Specifically, the peripheral object ID setting unit 1605, for each object whose edge is extracted in step ST1702, is based on the reference position coordinates of each object set in step ST1703, and other objects within a preset range. And the object ID of the specified object is set as the peripheral object ID.

  The reference position information setting unit 1606 of the reference position information setting control unit 108 sets reference position related information to be recorded on the recording medium 104, and records the set reference position related information on the recording medium 104 (step ST1706). Specifically, the reference position information setting unit 1606 extracts the edge feature value and object ID extracted and set in step ST1702, the reference position coordinates set in step ST1703, the reliability group set in step ST1704, The peripheral object ID set in ST1705 is associated with the reference position related information, and the reference position related information is recorded on the recording medium 104.

The hardware configuration of the photographing direction variation detection apparatus 106 according to the second embodiment of the present invention is the same as the hardware configuration described with reference to FIG.
Reference position coordinate information acquisition unit 107, edge extraction unit 1601, reference position coordinate determination unit 1603, reliability group setting unit 1604, peripheral object ID setting unit 1605, and reference position information setting unit 1606 are HDD 1402, It is realized by a processing circuit such as a CPU 1401 that executes a program stored in the memory 1403 or the like, or a system LSI.
In addition, a plurality of processing circuits may cooperate to execute the above function.
Note that the processing order of steps ST1703 to ST1705 is not limited to this, and may be switched.

As described above, according to the second embodiment, the reference position coordinates recorded in the recording medium 104 are not registered in advance, and are real-time based on information constantly acquired from the reference coordinate related information setting camera 901. Since the reflection is reflected, the movement of the reference position coordinates to be compared with the position coordinates of the object on the captured image of the camera 100 to detect the change in the shooting direction can be reflected in the information on the recording medium 104 in real time. it can.
In addition, since fluctuations in the reference position coordinates can be reflected in real time, the position coordinates of a moving object such as a person, not a fixed object, can be used as reference position coordinates for comparison in the shooting direction fluctuation determination of the camera 100. Can be targeted.

In the present invention, within the scope of the invention, any combination of the embodiments, or any modification of any component in each embodiment, or omission of any component in each embodiment is possible. .
In the first embodiment, the photographing direction variation detection device 106 is configured as shown in FIGS. 1 and 2, but the photographing direction variation detection device 106 includes a position coordinate calculation unit 201, a position coordinate comparison unit 202, and the like. By providing the fluctuation amount calculation unit 204, the above-described effects can be obtained.

  100, 901 Camera, 101 Video signal acquisition unit, 102 Video signal processing unit, 103 Direction variation determination unit, 104 Recording medium, 105 Alarm unit, 106 Shooting direction variation detection device, 107 Reference position coordinate information acquisition unit, 108 Reference position Information setting control unit, 201 position coordinate calculation unit, 202 position coordinate comparison unit, 203 reference position coordinate change presence / absence confirmation unit, 204 fluctuation amount calculation unit, 1401 CPU, 1402 HDD, 1403 memory, 1404 speaker, 1405 display, 1406 communication I / F device, 1601 edge extraction unit, 1603 reference position coordinate determination unit, 1604 reliability group setting unit, 1605 peripheral object ID setting unit, 1606 reference position information setting unit.

Claims (7)

Based on the image captured by the imaging device, the position coordinates of the object on the image are calculated, and the calculated position coordinates are trusted according to the possibility of movement of the object with respect to the reference position coordinates to be compared with the position coordinates. A position coordinate calculation unit for performing grouping according to the reliability group set based on the degree;
The position coordinates calculated by the position coordinate calculation unit are compared with the reference position coordinates in order from the position coordinates grouped into a highly reliable group, and the position coordinates and the reference position coordinates match. A position coordinate comparison unit for determining whether or not
Based on the determination result of the position coordinate comparison unit, a difference between the position coordinate and the reference position coordinate is calculated, and when the calculated difference is equal to or greater than a threshold, a change in the shooting direction of the imaging device is detected. An imaging direction variation detection apparatus comprising: a variation amount calculation unit. When the position coordinate comparison unit determines that the position coordinates and the reference position coordinates do not match, the reference that is the comparison target according to the group in which the position coordinates determined not to match is grouped It further includes a reference position coordinate change presence / absence confirmation unit that determines whether or not the position coordinates are changed,
The fluctuation amount calculation unit
When the position coordinate comparison unit determines that the position coordinate does not match the reference position coordinate, and the reference position coordinate change presence / absence confirmation unit determines that the reference position coordinate is not changed, the position coordinate The imaging direction variation detection device according to claim 1, wherein when the calculated difference is equal to or greater than a threshold value, a variation in the imaging direction of the imaging device is detected. . A video signal processing unit that detects an object on the image captured by the imaging device and extracts an edge of the detected object;
The position coordinate calculation unit
The imaging direction variation detection device according to claim 1, wherein the position coordinates of the object on the image are calculated based on the feature amount of the edge extracted by the video signal processing unit. The alarm part which outputs a warning when the fluctuation amount calculation part detects the fluctuation of the photographing direction of the imaging device, further comprises an alarm part. Photographic direction fluctuation detector. The reference position coordinate change presence confirmation unit is
The imaging direction variation detection device according to claim 2, wherein, when it is determined that the reference position coordinates are changed, the reference position coordinates are updated to the contents of the position coordinates subjected to the comparison. An information acquisition unit for reference position coordinates for acquiring an image taken by an imaging device different from the imaging device;
An edge extraction unit that detects an object on the image acquired by the reference position coordinate information acquisition unit and extracts an edge of the detected object;
The reference position coordinate determination unit that sets the reference position coordinate based on the information on the edge of the object on the image extracted by the edge extraction unit. The photographing direction variation detection device according to any one of the preceding claims. A position coordinate calculation unit calculates a position coordinate of an object on the image based on an image captured by the imaging device, and the object moves with respect to a reference position coordinate to be compared with the calculated position coordinate. Perform grouping according to the reliability group set based on the reliability according to the possibility,
The position coordinate comparison unit compares the position coordinates calculated by the position coordinate calculation unit with the reference position coordinates in order from the position coordinates grouped into a highly reliable group, and the position coordinates and the reference coordinates Determine if the position coordinates match,
A fluctuation amount calculation unit calculates a difference between the position coordinate and the reference position coordinate based on the determination result of the position coordinate comparison unit, and when the calculated difference is equal to or greater than a threshold value, the imaging of the imaging device An imaging direction variation detection method characterized by detecting a variation in direction.

Images