JP5147761B2 - Image monitoring device - Google Patents

Description

  The present invention relates to an image monitoring apparatus that processes an image captured in a monitoring space to detect the position of a person in the monitoring space, and more particularly to an image monitoring apparatus that detects the position of a person from images captured by a plurality of imaging units. About.

  In order to monitor a person's behavior over a wide range, a plurality of cameras are installed, the position of the person is detected from each of the images captured by each camera, and the information of the person position detected from each image is integrated. Things have been done.

  In order to integrate information on person positions, in Patent Document 1, pieces of information with close positions of the center of gravity are integrated as information on the same person (paragraphs 0015 and 0033).

JP 7-49952 A

  However, the detected person position generally includes an error, and the detection error causes erroneous integration. There are various factors of detection error, such as occlusion, shadow, quantization error, etc. in which a part or all of the captured human image is hidden by another person or an obstacle. In particular, the detection error due to occlusion was large, which was the main cause of erroneous integration.

  For example, FIG. 13A shows a state in which the lower body part of the person 90 is concealed due to the influence of the occlusion by the obstacle 91 and the center of gravity of the upper body part of the person 90 is detected as the center of gravity position 92 of the person 90. The center of gravity of the person 90 should be detected at the position of the center of gravity 93 in the region where the upper body and the lower body are combined.

  If a detection error is included, person positions that are originally the same person are separated from each other, or the separated person positions are close to other person positions. Therefore, in the prior art, there is a problem that person positions originally belonging to the same person are made different persons or person positions originally belonging to other persons are made to be the same person.

  Further, comparison of person positions detected from images taken by different cameras is performed after projecting each person position onto a common coordinate system. When projecting, it is necessary to set the height of each person, and an error in setting the height was one of the factors that caused erroneous integration.

  The present invention has been made in view of the above-described problem, and even if an error is included in a person position detected from images picked up by a plurality of image pickup units, the person positions of the same person are collected together without error. An object of the present invention is to provide an image monitoring apparatus capable of integrating position information.

  The inventor of the present application has studied to realize an image monitoring apparatus that solves the above problems, and as a result, the detection error of a vertically long person appears larger in the height direction than the width direction of the person. It was found that the direction detection error appears on the projection line from the imaging unit to the person. For example, the person position 92 detected in the example of FIG. 13A is regarded as the barycentric position 95 of the person image 94 present at a position away from the imaging unit along the projection line 96 as shown in FIG. It will be done. Since this can be said for each of a plurality of camera systems, the inventor of the present application is true at a position where projection lines to a person position detected from images captured by different imaging units intersect even if errors are included. It is derived that the person position is most likely to exist. The present invention utilizes this finding.

  An image monitoring apparatus according to the present invention includes a plurality of imaging units that share a part of a field of view and capture a person moving in the field of view from different directions and output a monitoring image, and a person from the monitoring image. Person position detection means for detecting a person position in a local coordinate system, and coordinate conversion means for projecting the person position from a local coordinate system set for each imaging unit to a common coordinate system set in common for a plurality of imaging units And a projection line calculating means for calculating a projection line connecting a person position in the common coordinate system and a position of the imaging unit that captured the monitoring image where the person position was detected, and a monitoring image captured simultaneously by a plurality of imaging units A pair of a person position detected from one of the above and a person position detected from the other monitoring image is set, and the person position passing through the projection line to one person position and the projection line to the other person position The path length calculation means that calculates and outputs the path length of the person, and the pair of shorter path lengths output by the path length calculation means is preferentially collected as the person position of the same person over the other pairs. And a person position integration means for integrating information.

  In the case of a pair of the same person, the projection lines to these person positions intersect at a position relatively close to the person position, so a relatively short path length is calculated even if a detection error is included. On the other hand, in the case of a pair of others, a long path length is calculated because the projection lines intersect at a position away from the person position due to a contradiction in the combination. Thus, according to the above configuration that integrates the information of the person position based on the path length in which a significant difference appears between the pair of the same person and the pair of others, the person position of the same person even if a detection error occurs in the person position It is possible to integrate the information on the person position by appropriately gathering.

Further, in a preferred aspect of the present invention, the coordinate conversion means performs projection by setting a plurality of heights within a person's height distribution range, and the path length calculation means includes a path length calculated for each of the plurality of heights. Output the minimum path length.
According to such a configuration, errors due to height setting errors are reduced and the path length for the same person pair is minimized, so the difference between the same person pair and the other person pair becomes more conspicuous, and the integration accuracy Will improve.

Further, in a preferred aspect of the present invention, the coordinate conversion means further back-projects the intersection of the projection lines to each person position constituting the pair to the local coordinate system where the person position is detected, and the path length calculation means includes The sum of the distance between the person position detected by the person position detecting means and the back-projected intersection is calculated as the path length.
According to this configuration, the intersection of the same person pair exists near these person positions, and the intersection point and the person position have the same resolution in the local coordinate system, so the path length with the resolution difference corrected is calculated. As a result, it is possible to integrate the information of the person position with higher accuracy.

Further, in a preferred aspect of the present invention, the coordinate conversion means sets a plurality of heights within a person's height distribution range and back-projects the intersections, and the path length calculation means calculates for each of the plurality of heights. Outputs the minimum path length among the path lengths.
According to such a configuration, since the path lengths can be compared in a state where the error due to the height setting error and the resolution disparity are reduced, the information on the person position can be integrated with higher accuracy.

  Also, in a preferred aspect of the present invention, the person position integration means calculates the intersection of the projection lines on the pair that is the same person as the position after the integration of the person, so that one person position of the same person is obtained. Can be summarized.

  According to the present invention, even if a detection error occurs in the information on the person position detected from the images picked up by the plurality of image pickup units, the information on the person position can be integrated appropriately.

1 is an overall configuration diagram of an image monitoring apparatus. It is a figure which shows the example of arrangement | positioning of each part of an image monitoring apparatus. It is the figure which illustrated a mode that a person position was detected. It is the figure which illustrated a mode that a path length was calculated. It is another figure which illustrated a mode that a path length was calculated. It is the figure which illustrated function f (*) which calculates proximity. It is a figure which shows the example of calculation of a proximity and an individuality. It is a figure which shows the example of calculation of an integrated evaluation value. It is a flowchart of an image monitoring process. It is a flowchart of person information integration processing in the first embodiment. It is a flowchart of the person information integration process in 2nd embodiment. It is the figure which illustrated a mode that path length was calculated in a local coordinate system. It is the figure which illustrated the detection error of the person position.

  As an example of a preferred embodiment of the present invention, an image monitoring apparatus that detects a suspicious person by analyzing a movement trajectory of a person moving in a monitoring space will be described.

<First embodiment>
[Configuration of image monitoring device]
The configuration of the image monitoring apparatus 1 according to the first embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a functional block diagram, and FIG. 2 shows an arrangement example.

  The image monitoring apparatus 1 includes a plurality of camera units 2 and an integrated unit 3 connected via a communication network 4 such as a LAN. Each of the camera units 2 detects the position (person position) of a person existing in its own field of view, and transmits the detected person position to the integrated unit 3. The integration unit 3 integrates the information on the person position received from each camera unit 2. At this time, person positions by the same person are combined into one. The integration unit 3 recognizes the behavior of the person based on the integrated person position and detects suspicious behavior. The image monitoring apparatus 1 can be monitored over a wide range and with few blind spots by integration. In addition, since it is based on the information of the person position, there is an advantage that there is no need to perform color calibration between the camera units 2 and the communication amount between the camera unit 2 and the integrated unit 3 is small. It is assumed that the units 2 and 3 are synchronized with each other and operate in synchronization.

  Each of the camera units 2 includes an imaging unit 20, a camera storage unit 21, and a camera communication unit 23 connected to the camera control unit 22.

  The imaging unit 20 is a so-called surveillance camera. Each imaging unit 20 captures a monitoring space at regular intervals and outputs a monitoring image. Each imaging unit 20 is installed so as to be separated from at least an adjacent imaging unit 20 so as to share a part of the field of view 50, and images a person moving in the field of view simultaneously from different directions. As a result, a monitoring image in which no occlusion has occurred can be simultaneously obtained from another imaging unit 20 for a person in which occlusion has occurred in the monitoring image captured by one imaging unit 20. Preferably, each imaging unit 20 is configured by a surveillance camera having a fisheye lens, and is installed with its optical axis directed vertically downward. Preferably, a setting is made in advance so that an imaging range in which a resolution capable of detecting a person position is obtained by image processing is regarded as each visual field 50 among the imaging ranges of the imaging units 20.

  The camera storage unit 21 is a memory device such as a ROM (Read Only Memory) or a RAM (Random Access Memory). The camera storage unit 21 stores in advance the address of each unit and various programs, and also represents data such as a template representing an image characteristic (color histogram) of a person in the field of view, a monitoring image, and a background image when no person is in the field of view. Is stored as necessary.

The camera control unit 22 includes a DSP (Digital Signal Processor), MCU (Micro Control
Unit). The camera control unit 22 functions as each unit by reading out a program describing the operation of the person position detection unit 220 and the like from the camera storage unit 21 and executing it.

  The person position detection unit 220 detects a person position where a person is present from the monitoring image, and outputs the detected person position to the camera communication unit 23. The person position is detected in an exploratory manner as follows. The person position detection unit 220 extracts a change area by performing a difference process between the background image and the monitoring image, sets a candidate position in the change area, and has the same shape and size as the template having the candidate position as the center of gravity. And the similarity between the color histogram extracted from the monitoring image and the color histogram of the template in the area is calculated. The calculation of the similarity is repeated while sequentially moving the candidate positions, and the person position detection unit 220 detects the candidate position where the similarity that is the maximum and equal to or higher than a preset reference value is calculated as the person position.

  As described above, the position of the center of gravity of the person image is detected as the person position. In another embodiment, the uppermost position of the person image assuming the top of the head, the lowermost position of the person image assuming the foot, or the like may be employed as the person position. However, the uppermost position and the lowermost position, which are the outer edges of the human image, are easily affected by the error in extracting the change area, and the uppermost position has a large projection error, which will be described later, because the projection angle of the projection line is small The position tends to be lost due to occlusion, and the extraction error is large. On the other hand, the position of the center of gravity is superior to the uppermost position and the lowermost position in that the influence of these extraction errors and projection errors is diluted and becomes smaller.

  In addition, the person position detection unit 220 updates the background image to be replaced with the monitoring image when the change area is not extracted, updates the template that newly adds the image feature of the change area that does not match any template, and matches any change area. Data update such as update to discard the template not to be performed is performed as appropriate.

  The camera communication unit 23 is a communication circuit suitable for the communication network 4. The camera communication unit 23 is connected to the communication network 4 and transmits the person position input from the person position detection unit 220 to the integrated unit 3 via the communication network 4.

  The integrated unit 3 includes an integrated communication unit 30, an operation unit 31, an integrated storage unit 32, and an output unit 34 connected to an integrated control unit 33.

  The integrated communication unit 30 is a communication circuit suitable for the communication network 4. The integrated communication unit 30 is connected to the communication network 4 and receives a person position from each of the camera units 2 via the communication network 4.

  The operation unit 31 is a user interface device such as a touch panel display. Used by the administrator of the image monitoring apparatus 1 to input various settings.

  The integrated storage unit 32 is a memory device such as a ROM or a RAM. The address of each unit and various programs are stored in advance, and various data are stored as necessary. The various data includes the installation position CAM of the imaging unit 20, the projection rule 320, and the movement locus 321.

  The projection rule 320 is a coordinate conversion formula for coordinate-converting the person position between the local coordinate system of the monitoring image plane set for each imaging unit 20 and the common coordinate system set in common for all the imaging units 20. is there.

  The local coordinate system of the imaging unit 20-1 is represented by an x1 axis and a y1 axis that are orthogonal to each other on the imaging surface 51-1. Similarly, the local coordinate system of the imaging unit 20-2 is represented by an x2 axis and a y2 axis that are orthogonal to each other on the imaging surface 51-2.

  In the common coordinate system, a horizontal plane in the three-dimensional world coordinate system representing the actual monitoring space 52 is set. That is, the world coordinate system is represented by the X axis, the Y axis, and the Z axis orthogonal to each other, and the XY plane when the Z axis is set in the vertical direction of the monitoring space 52 is set as the common coordinate system. Although the Z coordinate of the plane is arbitrary, in this embodiment, the plane is set to Z = 0 in order to simplify the processing.

  The projection rule 320 is set for each image capturing unit 20 by applying a camera model such as a known pinhole camera model to the camera parameters of each image capturing unit 20. The camera parameters are the focal length, angle of view, number of pixels and lens distortion of the imaging unit 20, the installation position CAM and imaging direction of the imaging unit 20 in the world coordinate system, and are input from the operation unit 31 prior to image monitoring. .

  Further, in order to project a person position in the local coordinate system to one point in the common coordinate system, it is necessary to set the height of the person and use the projection rule 320. This height is appropriately set by coordinate conversion means 330 described later.

  The movement locus 321 is a history of person positions in a common coordinate system integrated for each person.

  The integrated control unit 33 is an arithmetic device such as a CPU (Central Processing Unit) or a DSP. The integrated control unit 33 stores programs describing the operations of the coordinate conversion unit 330, the projection line calculation unit 331, the path length calculation unit 332, the person position integration unit 333, the tracking unit 334, the suspicious behavior detection unit 335, and the like. It functions as each means by reading from and executing.

  The coordinate conversion means 330 uses the projection rule 320 to coordinate the person position between the local coordinate system and the common coordinate system. Specifically, when the person position in the local coordinate system detected by the person position detecting unit 220 is input to the coordinate conversion unit 330, a predetermined height is set, and the person position is projected onto the common coordinate system and then projected. The person position is output. The coordinate conversion means 330 sets the height within the distribution range of the group of persons whose monitoring targets. For example, if the subject is an adult subject such as an ATM corner, the coordinate conversion means 330 sets the height in the range of 150 cm to 200 cm.

  The projection line calculation means 331 generates a straight line connecting the person position in the common coordinate system and the installation position CAM of the imaging unit 20 that captured the monitoring image where the person position is detected, that is, a projection line from the imaging unit 20 to the person position. calculate.

  The path length calculation unit 332 sets a pair of a person position detected from one of the monitoring images simultaneously captured by the plurality of imaging units 20 and a person position detected from the other monitoring image, and one person position of the pair The path length between the person positions passing through the projection line and the projection line to the other person position of the pair is calculated and output to the person position integration unit 333. The calculated path length represents the closeness between the positions of persons located at both ends of the path, and is used as a scale for determining whether or not these person positions belong to the same person.

  Since the moving person takes a standing posture, the image is captured in a vertically long shape, that is, a shape whose height is larger than the width. Therefore, the component in the height direction is dominant in the detection error of the person position. The error component in the height direction appears in a direction away from the imaging unit 20, that is, a direction along a projection line from the imaging unit 20 to the person position.

  For this reason, it is likely that the possibility of the presence of a true person position is high in the direction along the projection line. When the detection result of the person position in the two camera units 2 is obtained, the possibility that the true person position exists is likely to be the highest at the intersection of the projection lines to each person position. . The above path length is the sum of the distance between one of the pair and the intersection, and the distance between the other of the pair and the intersection, and the detection error based on the position where the possibility that the true person position is most likely exists is used. Represents.

  In a pair that is originally the same person, the intersection is calculated near the position of the person making up the pair, so the path length is calculated to be relatively short. However, in the case of a pair who is originally a stranger, there is a contradiction in which the positions of other persons are forcibly paired, so the intersection is calculated at a position far from the position of the person making up the pair, and the path length via the intersection is become longer. Therefore, it is possible to accurately identify the same person pair and the other person pair by referring to the path length having such properties.

  Incidentally, in the prior art in which the distance between person positions is evaluated as a detection error, it is considered that there is a high possibility that a true person position exists at an intermediate position between detected person positions. Although it may be good to approximate the detection error of the pair who is originally the person, the possibility of erroneous evaluation is increased because an unreasonably small distance is calculated even for the pair which is originally the other person.

  With reference to FIG. 4, how the path length is calculated will be described with a specific example.

  The person α exists in the shared visual field of the imaging unit 20-1 and the imaging unit 20-2. For the person α, the person position a1 is detected in the local coordinate system of the monitoring image 60-1 captured by the imaging unit 20-1, and the person α is detected in the local coordinate system of the monitoring image 60-2 captured by the imaging unit 20-2. A position a2 is detected. However, in the monitoring image 60-1, the occlusion due to the obstacle 53 has occurred, and the person position a1 has been detected with a deviation from the original person position in the head direction. The person β exists only in the field of view of the imaging unit 20-2. A person position b2 is detected for the person β in the local coordinate system of the monitoring image 60-2.

  The person positions a1, a2, and b2 are respectively projected by the coordinate conversion means 330 onto the person positions A1, A2, and B2 in the common coordinate system. Due to the influence of the detection error, A1 moves away from A2 and approaches B2, and there is almost no difference between the distance between A1 and A2 and the distance between A1 and B2.

  In the common coordinate system, the projection line calculation unit 331 calculates the projection line CAM1-A1 to A1, the projection line CAM2-A2 to A2, and the projection line CAM2-B2 to B2. Λ1 represents the intersection of the projection line CAM1-A1 and the projection line CAM2-A2, and Λ2 represents the intersection of the projection line CAM1-A1 and the projection line CAM2-B2. D1 represents a distance between A1 and Λ1, D2 represents a distance between A2 and Λ1, D3 represents a distance between A1 and Λ2, and D4 represents a distance between B2 and Λ2. The path length calculation means 332 calculates the path length δ between A1 and A2 as (D1 + D2) and the path length δ between A1 and B2 as (D3 + D4).

  The intersection Λ1 related to the person positions A1 and A2 that are originally of the same person α is calculated at a position relatively close to the person positions A1 and A2. In particular, the distance D2 between the person position A2 where no occlusion occurs and the intersection Λ1 is very short. On the other hand, the intersection Λ2 relating to the person positions A1 and A2 that are originally belonging to another person is calculated at a position greatly deviating from both A1 and A2. Therefore, even in the case of this example in which there is almost no difference between the distance between A1 and A2 and the distance between A1 and B2, the path length between A1 and A2 (D1 + D2) and the path length between A1 and B2 (D3 + D4) A noticeable difference appears.

  By referring to the path length thus calculated, it is possible to easily determine that A1 and A2 having a short path length, that is, the pair of person positions a1 and a2 are the same person.

  Here, the path length between person positions that are originally the same person includes an estimation error of height in addition to an error due to occlusion. The coordinate conversion means 330 defines the average height and performs projection. However, since the height of the person is actually different, an estimation error occurs. For example, in the example of FIG. 4, the position of a person α having a height of 185 cm is projected with an average height of 170 cm. An estimation error of height appears at the distance D2, and an error due to occlusion and an estimation error of height appear in combination at the distance D1.

  Therefore, the path length calculation unit 332 sets a plurality of heights within the height distribution range of the person being monitored for each pair of person positions, calculates the path length for each height, and sets the minimum path length to The path length of the pair. As a result, the path length in which the estimation error of the height is reduced is calculated, so that the person position of the same person can be more accurately identified.

  FIG. 5A shows how the path length is calculated when the height is 185 cm. This setting is a setting for calculating the minimum path length in the pair of person positions A1 and A2 that are originally the same person. Both the distances D2 and D1 decrease and become 0 when reaching the distance D2. FIG. 5B shows how the path length is calculated when a height of 150 cm is set. This setting is a setting when the minimum path length is calculated for the pair of person positions A1 and B2, which is originally belonging to another person. Although both the distances D3 and D4 are decreasing, there is a limit to the decrease within the height distribution range, and the path length is not shortened as in the case of the same person. In another case, the distance D3 decreases but the distance D4 increases, or the distance D4 decreases but the distance D3 increases, and the path length is not as short as the case of the same person.

  By searching for height in this way, the path length related to the same person is minimized, and the difference from the path length related to other persons can be easily opened. Therefore, the person position of the same person can be more accurately identified based on the path length. It becomes possible.

  The person position integration unit 333 compares the path lengths calculated by the path length calculation unit 332 and summarizes pairs with shorter path lengths as person positions of the same person preferentially over other pairs. Are integrated, and the integration result is output to the tracking means 334. At this time, the person position integration unit 333 performs integration by exclusively combining pairs so that one-to-many integration (for example, simultaneous establishment of a pair of person positions A1 and A2 and a pair of person positions A1 and B2) does not occur. The person position integration means 333 combines the person positions of the same person into one by setting the intersection of the projection lines to the pair of person positions determined by the same person as the integrated person position. Note that for a single person position that is not grouped by the same person, the person position integration unit 333 sets the person position as it is as the person position after integration.

  The person position integration unit 333 compares the path lengths by converting the path lengths into proximity degrees representing the proximity between the person positions and comparing the proximity degrees. The greater the proximity, the shorter the path length. The conversion is performed by inputting the path length δ to a function f (·) set in advance so that a larger value is output as the path length δ is shorter. A function 70-1 shown in FIG. 6 is a specific example of the function f (•). That is, the function 70-1 outputs the upper limit value when δ = 0, the output decreases as δ increases when 0 ≦ δ <Δ, and outputs the lower limit value when δ ≧ Δ. In another embodiment, a function f (·) that is weighted with an interval 0 ≦ δ <Δ is set as in the function 70-2 or 70-3.

  Here, not all of the person positions detected by the person position detecting means 220 have a pair. That is, a person who is in the field of view of one imaging unit 20 but is not in the field of view of the other imaging unit 20, or a person who is in the field of view of both imaging units 20 but is hardly captured from one imaging unit 20 due to occlusion. The person position is detected only from one person position detecting means 220. Such person positions should not be integrated as one person position in the pair, but as a single person position.

Therefore, the person position integration unit 333 calculates a single degree representing a probability scale that the single person position is independent, and integrates the person positions having a single degree higher than the proximity without collecting them. The degree of isolation is calculated by the following formula. Note that 1.0 in the equation is the upper limit value of the proximity described above.
Individuality of the person position = 1.0−maximum value of the proximity regarding the person position (1)
For example, in the example of FIG. 7, the individuality 81 of the person position A1 is calculated as 1.0−max {0.8, 0.6} = 0.2. Similarly, the individuality 81 of the person positions A2 and B2 is calculated as 0.2 and 0.4, respectively.

  Specifically, the person position integration unit 333, when combining person positions as much as possible in order to integrate the person positions in consideration of proximity and singleness, leaves some person positions alone. Set a combination (hereinafter referred to as “integrated pattern”) that covers a pair of people positions and a case where all human positions are single, and select the proximity and / or singleness according to each integrated pattern Then, the integrated evaluation value is calculated by summing up and the integrated pattern for which the highest integrated evaluation value is calculated is selected.

  By determining the integration pattern in this way, pairs with higher proximity are preferentially grouped as person positions of the same person over other pairs, and person positions with higher individuality are integrated without being combined. Is done. As a result, pairs of lower proximity calculated are not forcibly collected, and information on person positions can be integrated with higher accuracy.

  For example, when one person position is detected in the camera unit 2-1 and two person positions are detected in the camera unit 2-2 as in the example of FIG. 3, three integrated patterns # are displayed as shown in FIG. The integrated evaluation values are calculated as 1.2, 0.8, and 0.8 for each of 1 to # 3, and the integrated pattern # 1 having the maximum integrated evaluation value is selected. That is, in the example shown in FIGS. 3 to 8, it is determined that “the person positions A1 and A2 are integrated as a person position of the same person and the person position B2 is integrated as a single person position”. In this determination, first, the pair of A1 and A2 having the highest value in proximity and singleness is selected first, and then the proximity and singleness related to A1 and A2 already selected are excluded. This means that the highest singleness of B2 is selected with priority on the second.

  The person position integration unit 333 calculates the intersection of the projection lines CAM1-A1 and CAM2-A2 as the position of the person α and the person position B2 as the position of the person β.

  The tracking unit 334 generates a movement trajectory 321 for each person by associating the integrated person positions output from the person position integration unit 333 at the preceding and following times based on the consistency of the person positions. Specifically, the tracking unit 334 predicts the person position of each person at the current time from the movement trajectory 321 generated up to one hour before, and within the range of the distance threshold set in advance for each predicted position. The closest integrated person position is added to the movement locus 321 as the person corresponding to the predicted position. The predicted position can be calculated by applying a motion model such as a constant velocity motion model or a prediction filter such as a Kalman filter to the movement locus 321.

  Note that the tracking unit 334 newly registers a person position that does not correspond to any predicted position as a newly appearing person in the movement locus 321, and calculates a predicted position that does not correspond to any person position. 321 is deregistered from the movement trajectory 321 because it is a person who has disappeared out of the field of view.

  The suspicious behavior detecting means 335 analyzes the movement trajectory 321 to determine the presence or absence of the suspicious behavior, and when detecting the suspicious behavior, the output unit 34 outputs an abnormal signal including the detection of the suspicious behavior and the movement trajectory 321 in which the suspicious behavior is detected. Output. The suspicious behavior is, for example, a stay exceeding a specified time, or a pre-set entry area or the like.

  The output unit 34 is a sound output device such as a speaker or a buzzer, a monitor device such as a CRT or a liquid crystal display, or a communication device that communicates with a security center or the like via a communication network. When the abnormal signal is input, the output unit 34 rings the sound output device to notify the monitoring staff of the detection, displays the movement locus 321 in which the suspicious behavior is detected on the monitor device, or to the security center. An abnormal signal is transmitted or an abnormal signal is output.

[Operation of image monitoring device]
The operation of the image monitoring apparatus 1 will be described with reference to FIG.

  When an administrator who confirms that the monitoring space is unmanned, powers on each unit and each unit is initialized and starts operating. The integrated control unit 33 checks whether the projection rule 320 is stored in the integrated storage unit 32 (S1), and if not stored (NO in S1), requests the administrator to input camera parameters. When camera parameters are input from the operation unit 31 by an input operation by the administrator (S2), the integrated storage unit 32 stores the projection rule 320 and the installation position CAM based on the camera parameters in the camera storage unit 32 (S3).

  That is, the integrated control unit 33 sets the projection rule 320 and the installation position CAM1 between the local coordinate system and the common coordinate system of the imaging unit 20-1 based on the input camera parameters of the imaging unit 20-1 to the camera unit 2-1. The camera unit 2 stores the projection rule 320 and the installation position CAM2 between the local coordinate system and the common coordinate system of the imaging unit 20-2 based on the input camera parameters of the imaging unit 20-2. -2 is stored in association with the address -2.

  If the projection rule 320 or the like has been set or has already been set (YES in S1), thereafter, the processing of S4 to S11 is repeated each time the imaging unit 20 captures a new monitoring image at regular time intervals. It is. In the following description, the unit of time recorded at the predetermined time interval is referred to as time.

  When a new monitoring image is input (S4), the person position detecting means 220 of each camera control unit 22 detects the person position from the monitoring image (S5) and transmits the detected person position to the integrated unit 3. . At this time, the person position detecting means 220 gives the address of the own unit 2 as the transmission source address and the address of the integrated unit 3 as the transmission destination address, and outputs them.

  The integration control unit 33 of the integration unit 3 that has received the information on the person position from each camera unit 2 integrates the information (S6).

  Details of the person information integration processing in step S6 will be described with reference to FIG.

  First, the path length calculation unit 332 of the integrated control unit 33 sequentially generates pairs of person positions and sets a loop process (S60 to S69), and calculates path lengths for all pairs. Here, it should be noted that the pair is composed of person positions detected from different monitoring images at the current time. For example, in the example of FIG. 3, two pairs of a1 and a2, a1 and b2 are sequentially generated.

  Further, the path length calculation means 332 sequentially sets the person's height h as 150 cm, 155 cm, 160 cm,..., 200 cm to set loop processing (S61 to S67), and calculates the path length for each pair of heights h. .

  In the loop processing, first, the coordinate conversion means 330 of the integrated control unit 33 projects the two person positions constituting the pair on the common coordinate system using the projection rule 320 (S62). At this time, the coordinate conversion means 330 specifies the local coordinate system of the conversion source from the transmission source address given to each person position, and projects using the projection rule 320 corresponding to the specified local coordinate system. Further, the coordinate conversion means 330 performs projection by setting the height of each person position to h / 2. The reason for h / 2 is that the position of the person is the position of the center of gravity. The projection result is temporarily stored in association with the original person position.

  Next, the projection line calculation means 331 of the integrated control unit 33 calculates a projection line to each person position after projection (S63). At this time, the projection line calculation means 331 specifies the installation position CAM used for calculation of the projection line from the transmission source address given to each person position.

  Subsequently, the path length calculation unit 332 calculates the intersection of the two projection lines calculated in step S63 (S64), and calculates the distance between the projected person position calculated in step S62 and the intersection ( S65), the sum of these distances is calculated as the path length (S66).

  When the path length when each of the plurality of heights is assumed is calculated in this way (YES in S67), the path length calculation unit 332 selects the minimum path length among these path lengths for the processing target pair. Selected as the length δ (S68).

  When the minimum path length is selected for all pairs (YES in S69), person position integration means 333 of integrated control unit 33 inputs path length δ of each pair into function f (•) and approaches it. The degree f (δ) is calculated (S70), and the degree of isolation for each person position is calculated by applying the formula (1) to the proximity f (δ) (S71). An integrated evaluation value for the pattern is calculated (S72).

  Then, the person position integration unit 333 determines the integrated pattern for which the highest integrated evaluation value is calculated as a likely integrated pattern (S73), and integrates the person positions according to the determined integrated pattern (S74). In other words, the person position determined as a pair is calculated as the intersection of the projection lines to the person position, and the person position determined as independent is calculated as the person position integrated with the person position. In addition, you may divert the calculation result in step S64, without calculating an intersection again.

  Returning to FIG. 9, the tracking unit 334 of the integrated control unit 33 associates the person position of the integrated current time with the movement locus 321 up to one hour based on the consistency of the position, and the current time according to the association result. The tracking is advanced by updating the movement locus 321 at the person position (S7).

  When tracking progresses, the suspicious behavior detection means 335 of the integrated control unit 33 analyzes the updated movement trajectory 321 to determine the presence or absence of suspicious behavior (S8), and when suspicious behavior is detected (YES in S9), An abnormal signal including the movement locus where the suspicious behavior is detected is generated and output to the output unit 34 (S10). The output unit 34 to which the abnormality signal is input sounds the sound output means and displays the movement locus on the monitor device, or transfers the abnormality signal to the remote security center by the communication device, and the sound output device or monitor of the security center. Display is performed on the apparatus (S11). The observer who sees and listens to these displays determines the necessity of countermeasures such as guidance or exclusion, and if necessary, promptly approaches the surveillance space to take countermeasures.

  When the above process is completed, the process returns to S4 again.

<Second Embodiment>
On the monitoring image, the resolution of the person image is not uniform, and the resolution for the person far from the imaging unit 20 is lower than that of the near person. This means that even if the positions of persons detected for the same person are different, the position of the person detected by the camera unit 2 far from the person is more local than the position of the person detected by the camera unit 2 close to the person. This means that the detection error of one pixel in the coordinate system becomes larger in the common coordinate system.
Hereinafter, as a second embodiment of the present invention, an image monitoring apparatus 1 that calculates a path length in consideration of a resolution difference due to distance from the imaging unit 20 will be described. In addition, below, the part which is common in 1st embodiment is abbreviate | omitted and demonstrated.

[Configuration of image monitoring device]
The configuration of the image monitoring apparatus 1 according to the present embodiment will be described with reference to the configuration of FIG. 1 common to the first embodiment.

  In the first embodiment, the coordinate conversion unit 330 projects the person position from the local coordinate system to the common coordinate system. Further, when the intersection of the projection lines expressed in the common coordinate system is input, the coordinate conversion unit 330 in the second embodiment back-projects the intersection to the local coordinate system, and outputs the position of the intersection after the reverse projection.

  In the first embodiment, the path length calculation unit 332 calculates the path length in the common coordinate system. The route length calculation means 332 in the second embodiment calculates the route length in the local coordinate system instead of the common coordinate system. For this purpose, the path length calculation means 332 obtains the distance between the person position detected by the person position detection means 220 and the intersection position after back projection in the local coordinate system, and calculates the sum of the distances as the path length.

  The rest of the configuration in the second embodiment is the same as in the first embodiment.

[Operation of image monitoring device]
Among the operations of the image monitoring apparatus 1 in the second embodiment, the overall flow of the image monitoring process is the same as that of the first embodiment described with reference to FIG. The second embodiment and the first embodiment differ in operation in the details of the person information integration process in step S6.

  Hereinafter, the personal information integration process in the second embodiment will be described with reference to FIGS. 11 and 12.

  First, the coordinate conversion means 330 of the integrated control unit 33 uses the projection rule 320 to project all person positions detected by the person position detection means 220 at the current time onto a common coordinate system (S75). At this time, the coordinate conversion means 330 specifies the local coordinate system of the conversion source from the transmission source address given to each person position, and projects using the projection rule 320 corresponding to the specified local coordinate system. The coordinate conversion means 330 performs projection by setting the height of each person position to 1/2 of the average height, that is, 85 cm. The projection result is temporarily stored in association with the original person position.

  Next, the projection line calculation unit 331 of the integrated control unit 33 calculates projection lines to all the positions of the person after projection (S76). At this time, the projection line calculation means 331 specifies the installation position CAM used for calculation of the projection line from the transmission source address given to each person position. The calculation result is temporarily stored in association with the original person position.

  Subsequently, the path length calculation unit 332 of the integrated control unit 33 sequentially generates projected person position pairs and sets loop processing (S77 to S86), and calculates path lengths for all pairs.

In the loop processing for a pair, firstly the path length calculation unit 332 calculates the intersection of the two projection lines calculated for each person position pairs of the calculated projection line at step S 76 (S78).

  Subsequently, the path length calculation unit 332 sequentially sets the person's height h as 150 cm, 155 cm, 160 cm,..., 200 cm and sets a loop process (S79 to S84), and calculates the path length for each height h.

In the loop processing for the height h, firstly the path length calculation unit 332 three-dimensional coordinates of the intersection by assigning the height h / 2 at the intersection calculated in step S 78 (S80). If the coordinates of the intersection point are (xc, yc), the coordinates of the intersection point converted into a three-dimensional coordinate are set to (xc, yc, h / 2).

  Subsequently, the coordinate conversion means 330 back-projects the intersection point converted into the three-dimensional coordinates onto each of the local coordinate systems where the two person positions from which the intersection point is calculated are detected (S81), and the back-projected intersection point position Is output. At this time, the coordinate conversion means 330 identifies the local coordinate system of the conversion destination from the transmission source address given to the data of each person position constituting the pair, and reverses it using the projection rule 320 corresponding to the identified local coordinate system. Project.

  For example, when h = 170 cm is set for the pair of A1 and A2, the intersection Λ1 in FIG. 4 is x1-y1 which is the local coordinate system of the imaging unit 20-1 in which the person position a1 is detected as shown in FIG. The projection is back-projected to the position λ1 on the plane, and further back-projected to the position λ1 on the x2-y2 plane which is the local coordinate system of the imaging unit 20-2 where the person position a2 is detected.

Then the path length calculation unit 332, as a distance calculates (S82), the path length of these sums between the person position respectively and the intersection of the calculated original reverse the projected intersection at Step S 81 Calculate (S83).

  In the example of FIG. 12, the distance d1 between the person position a1 and the intersection λ1 and the distance d2 between the person position a2 and the intersection λ1 are calculated, and the path length is calculated as (d1 + d2).

  In the case of a pair of person positions by the same person, the intersection point is projected back to the vicinity of these person positions, so the intersection point and the person position have the same resolution in the local coordinate system. Since the path length can be calculated by calculating the intersection and the person position expressed with the same resolution, the path length in which the resolution disparity is corrected can be calculated.

  When the path length when each of the plurality of heights is assumed is calculated in this way (YES in S84), the path length calculation unit 332 calculates the minimum path length among these path lengths for the pair to be processed. Selected as the length δ (S85).

  The processing in steps S87, S88, S89, S90, and S91 after the path length is calculated for all pairs is the same as the processing in steps S70, S71, S72, S73, and S74 described with reference to FIG. It is.

  When the above process is completed, the process returns to step S7 in FIG.

  As described above, according to the second embodiment, the path length can be compared in a state where the resolution disparity is reduced by calculating the path length in the local coordinate system. Information can be integrated.

<Other variations>
In the embodiment described above, the person position integration unit 333 calculates the integrated evaluation value by calculating the singleness and taking the proximity and singleness into consideration. In another embodiment, the person position integration unit 333 calculates the integrated evaluation value without using the single degree. That is, the person position integration unit 333 sets an integrated pattern by excluding a pair whose proximity is lower than a preset proximity threshold and sets the integration pattern according to the set integration pattern. To calculate the integrated evaluation value.

  In the above embodiment, two camera units 2 are connected to the integrated unit 3. In another embodiment, a third camera unit 2-3 is connected to the integrated unit 3. In this case, the integration unit 3 integrates the person positions from the camera unit 2-1 and the camera unit 2-2, and then further integrates the integration result and the person position from the camera unit 2-3 to obtain three camera units. The person positions from 2 are integrated. Even when four or more camera units 2 are connected, person positions from all the camera units 2 can be integrated by performing integration in multiple stages in the same manner.

  In the above embodiment, the path length is calculated for each of the plurality of heights for each pair, and the shortest path length is calculated as the path length of the pair. In another embodiment, the processing load is reduced by fixing the height setting and calculating the path length in the height setting as a pair path length. In this case, in order to minimize the height estimation error, the height setting may be fixed to the average height or the most frequent height to be monitored.

In the above embodiment, the person position is detected by each camera unit 2, but in another embodiment, the person position is detected by the integrated unit 3. That is, the person position detection unit 220 is realized by the integrated control unit 33. In this case, since the monitoring image is transmitted and received, the communication load of the communication network 4 increases. However, it is not necessary to perform image processing in the camera control unit 22, and the procurement of the camera unit 2 is facilitated.

DESCRIPTION OF SYMBOLS 1 ... Image monitoring apparatus 2 ... Camera unit 3 ... Integrated unit 20 ... Imaging part 21 ... Camera storage part 22 ... Camera control part 23 ... Camera communication part 30 ... Integrated communication unit 31 ... operation unit 32 ... integrated storage unit 33 ... integration control unit 34 ... output unit 220 ... person position detection means 320 ... projection rule 321 ... movement locus 330 ... coordinate conversion means 331 ... projection line calculation means 332 ... path length calculation means 333 ... person position integration means 334 ... tracking means 335 ... suspicious behavior detection means

Claims (5)

A plurality of imaging units that share a part of the field of view and image a person moving within the field of view from different directions and output a monitoring image;
Person position detecting means for detecting a person position in a local coordinate system where the person exists from the monitoring image;
Coordinate transformation means for projecting the person position from the local coordinate system set for each imaging unit to a common coordinate system set in common to the plurality of imaging units;
A projection line calculating means for calculating a projection line connecting the position of the person in the common coordinate system and the position of the imaging unit that captured the monitoring image from which the position of the person was detected;
A pair of the person position detected from one of the monitoring images simultaneously captured by the plurality of imaging units and the person position detected from the other monitoring image is set, and the projection onto the one person position is performed. A path length calculation means for calculating and outputting a path length between the person positions passing through the projection line on the line and the other person position;
Person position integration means for integrating the information on the person position by combining the pair having a shorter path length output by the path length calculation means as the person position of the same person preferentially over the other pairs;
An image monitoring apparatus comprising: The coordinate conversion means performs the projection by setting a plurality of heights within the height distribution range of the person,
The image monitoring apparatus according to claim 1, wherein the path length calculation unit outputs a minimum path length among the path lengths calculated for the plurality of heights. The coordinate conversion means further back-projects the intersection of the projection lines to each person position constituting the pair to a local coordinate system where the person position is detected,
The image monitoring apparatus according to claim 1, wherein the path length calculation unit calculates a sum of distances between the person position detected by the person position detection unit and the back-projected intersection as the path length. The coordinate transformation means reversely projects the intersection by setting a plurality of heights within the height distribution range of the person,
The image monitoring apparatus according to claim 3, wherein the path length calculation unit outputs a minimum path length among the path lengths calculated for each of the plurality of heights. 5. The image monitoring apparatus according to claim 1, wherein the person position integration unit calculates an intersection of the projection lines with respect to the pair set as the same person as a position after integration of the person.