JP2014192776A - Video monitoring system, video processing server, and method for browsing and delivering monitored video - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video monitoring system for reducing the load of monitoring work and raising work efficiency, whether for real time monitoring or post browsing, by creating, from a large volume of video data shot by a plurality of cameras, an optimum browsing video for accurately and simply summarizing progression before and after the occurrence of a dangerous event.SOLUTION: A video monitoring system 10 comprises a plurality of cameras 20 for shooting a plurality of objects to be monitored moving within a monitoring district 50, video processing means 33, and a supervisory monitor 40. The video processing means calculates, on the basis of trajectory information about each object to be monitored, the intensity of group interaction between each object to be monitored changing along with the passage of time and, if there exist two or more objects to be monitored having a strong group interaction, classifies these objects to be monitored into one population. Next, the video processing means defines, on the basis of the intensity of group interaction, a cost for transition to each population by using a graph theory, and outputs a video constituted by a path for which a total cost is minimum to the supervisory monitor as an optimum browsing video.

Description

  The present invention relates to a video monitoring system, a video processing server, and a monitoring video browsing / delivery method that reduce the burden of monitoring work and increase work efficiency.

While there are concerns about the deterioration of global security, the market scale of industrial and private video surveillance systems is expanding rapidly, and in addition to the crime prevention field, disaster prevention, nursing care, management assistance (survey of visitor behavior), etc. It is used in a wide range of fields.
The video surveillance system transmits video from a number of surveillance cameras placed on the streets, facilities, stations, etc. to the monitoring room by wire or wirelessly, and the monitor in the monitoring room monitors the occurrence of a dangerous event by the monitor. is there.

With the development of high-speed network and high-performance storage in recent years, large-scale video surveillance using more than 100 surveillance cameras and switching the video of each surveillance camera to several tens of monitors arranged in parallel in the surveillance room A system has also appeared.
In such a large-scale video surveillance system, it is known that a contradiction occurs between an enormous amount of information and a human perception limit. That is, it is known that the work of constantly monitoring a large number of monitors places an excessive burden on the monitor, and is prone to human error due to fatigue and distraction.

Therefore, for example, in Patent Document 1, instead of a monitor, the video analysis system observes the movements and postures of the limbs of the person to be monitored, and displays the video on the monitor when it is determined as a dangerous event. A technique for reporting to a predetermined contact is disclosed. According to the present technology, there is an advantage that it is not necessary for the monitor to constantly monitor a large number of monitors.
Patent Documents 2 and 3 disclose techniques for expressing a monitoring target on a virtual three-dimensional space such as a monitor.
Patent Documents 4 and 5 disclose a technique for calculating a face direction from a moving direction of a person to be monitored, and automatically tracking the person with a camera at a position where the face can be photographed.
In Patent Documents 6 and 7, the position of the monitoring target and the position of the monitoring camera are recognized, and when the monitoring target moves out of the shooting range of one camera, the video is sequentially switched to the video of another camera. Is disclosed.

JP 2006-287883 A US Pat. No. 5,745,126 International Publication No. 2012/153805 U.S. Pat. No. 8,184,157 JP 2009-2730063 A U.S. Pat. No. 8,174,572 JP 2011-217320 A

However, the above prior art has the following problems.
That is, the techniques disclosed in Patent Documents 1 to 7 are basically for monitoring the situation at that time by so-called on-time (real-time monitoring). There is a problem that it is not suitable for the purpose of looking back (post review).
In daily life, the proportion of time during which a dangerous event occurs is very small relative to the total monitoring time. Therefore, after a dangerous event occurs, a supervisor or a related person such as the police (hereinafter collectively referred to as “monitorer”) grasps the monitoring video in order to grasp the reason for the occurrence of the dangerous event and the behavior of the suspicious person. When looking back, it is necessary to find a group of videos that may be related to the occurrence of a dangerous event from a huge amount of videos, for example, based on the estimated occurrence time of the dangerous event. There is a problem that it takes a lot of labor and time.

In addition, when a monitoring target such as Patent Documents 2 and 3 is expressed in a virtual three-dimensional space, data must be complemented when two-dimensional image data is synthesized into three dimensions. There is a problem that the reliability and authenticity of the video is not perfect, and the value as evidence is scarce.
In addition, when a plurality of cameras such as Patent Documents 4 and 5 automatically track a monitoring target, or when a monitoring target is tracked while appropriately switching images (cameras) such as Patent Documents 6 and 7, there are a plurality of monitoring targets. There is a problem that tracking is difficult when each monitoring target moves in a different direction, and that it is difficult to determine a monitoring target on the spot in real time monitoring.
As described above, it is difficult to efficiently perform both real-time monitoring and post-mortem browsing with the above-described conventional technology.

  In view of such a problem, the present invention creates real-time monitoring by creating an optimal browsing video that accurately and simply summarizes the course before and after the occurrence of a dangerous event from a large amount of video data captured by a plurality of cameras. In addition, it is an object of the present invention to provide a video monitoring system, a video processing server, and a monitoring video browsing / delivery method that reduce the burden of monitoring work and increase work efficiency in both cases of post-viewing.

The video monitoring system of the present invention is processed by a plurality of cameras that shoot a plurality of monitoring objects moving within a monitoring area, a video processing means that processes each video shot by the plurality of cameras, and the video processing means. A monitoring monitor for displaying the video, and the video processing means calculates the strength of the group interaction between the monitoring targets that changes with time based on the trajectory information of each monitoring target, and the group interaction If there are two or more monitoring targets that are strong, classify these monitoring targets as one population, and then use graph theory to determine the cost of transitioning to each population based on the strength of the group interaction. A video defined by a route having a minimum total cost is output to the monitoring monitor as an optimal browsing video.
In addition, when the monitor selects an arbitrary group or monitoring target, the video processing unit recalculates a path with the minimum total cost among the paths including the group or monitoring target, and the optimal viewing video Is output as
Further, the video processing means changes the playback speed of the optimal browsing video based on the strength of the group interaction.
Further, the video processing means processes and outputs the optimal browsing video so as to have a resolution corresponding to the performance of the monitor.

The video processing server of the present invention further includes video processing means for capturing a plurality of monitoring targets moving in the monitoring area with a plurality of cameras, processing each captured video, and outputting the processed video to a monitoring monitor. The processing means calculates the strength of the group interaction between the monitoring targets that changes over time based on the trajectory information of each monitoring target, and when there are two or more monitoring targets with strong group interaction. Classifies these monitoring targets as one individual group, then uses graph theory to define the cost of transitioning to each individual group based on the strength of the group interaction, and consists of paths that minimize the total cost. The video is output to the monitoring monitor as the optimum browsing video.
In addition, when the monitor selects an arbitrary group or monitoring target, the video processing unit recalculates a path with the minimum total cost among the paths including the group or monitoring target, and the optimal viewing video Is output as
Further, the video processing means changes the playback speed of the optimal browsing video based on the strength of the group interaction.
Further, the video processing means processes and outputs the optimal browsing video so as to have a resolution corresponding to the performance of the monitor.

Further, the monitoring video browsing / distribution method of the present invention includes a step of shooting a plurality of monitoring objects moving in the monitoring area with a plurality of cameras, a step of processing each video shot by the plurality of cameras, Displaying the recorded video on a monitoring monitor, and further, calculating a strength of group interaction between the monitoring targets that change with time based on the trajectory information of each monitoring target; When there are two or more monitoring targets with strong action, classify these monitoring targets as one individual group and the cost of transitioning to each individual group using graph theory based on the strength of the group interaction A step of defining, and a step of outputting a video composed of a route having a minimum total cost to the monitoring monitor as an optimal browsing video.
In addition, when the monitor selects an arbitrary population or monitoring target, a step of recalculating a route having the minimum total cost among the routes including the individual group or the monitoring target and outputting the route as an optimal viewing image It is characterized by providing.
In addition, the method includes a step of changing the playback speed of the optimal browsing video based on the strength of the group interaction.
In addition, the method includes a step of processing and outputting the optimum browsing video so as to have a resolution corresponding to the performance of the monitoring monitor.

According to the present invention, since the video processing means outputs only the optimum browsing video to the monitoring monitor, the supervisor does not need to monitor a large number of monitors, and the burden of monitoring work can be reduced.
Moreover, since there is a high possibility that the optimal browsing video includes the course before and after the occurrence of the dangerous event, the efficiency of the monitoring work can be improved.
In addition, by selecting an arbitrary individual group or a monitoring target when the monitor is viewing the optimal browsing video, a path with a minimum total cost is selected from the paths including the individual group or the monitoring target. Can be recalculated and output to the monitor, so the accuracy of the monitoring work can be improved.

Also, by changing the playback speed of the optimal browsing video, for example, for videos with a high probability of dangerous events, the playback speed is close to real time, and for videos with a low probability of dangerous events, increasing the playback speed, It is possible to shorten the playback time of the video with respect to the actual recording time, and it is possible to increase the efficiency of the monitoring work.
In addition, for example, when the monitor has a portable monitor monitor, an image corresponding to the performance of the monitor monitor can be obtained, for example, the optimal browsing image can be output with a reduced resolution.

Block diagram showing the configuration of the video surveillance system Flowchart showing processing until a continuous motion trajectory is generated The figure which shows the process at the time of producing | generating a continuous movement locus | trajectory (a)-(e) Flow chart showing the process until the optimal browsing video is output to the monitor A graph that schematically shows the motion trajectory of each monitoring target Flowchart showing processing when detecting group interaction Diagram showing processing when detecting group interaction Diagram showing the intensity distribution of group interaction The figure which shows the relationship between the movement state of two individuals (monitoring object) and group interaction Illustration for explaining hotspots Graph that superimposes individuals on the movement trajectory of each monitoring target Flow chart showing the process until the transition relationship diagram between each individual population is generated Graph showing the numbering of population formation time and elimination time Transition relationship diagram (a) and diagram (b) showing the path with the lowest total transition cost The figure which shows the state which adjusted the playback speed of the optimal browsing picture Figures (a) and (b) for explaining the recalculation of the optimal viewing video The figure which shows an example of the image layout of a monitoring monitor Is a diagram showing the actual image layout The flowchart which shows the process which a video processing means performs between the check work start of an optimal browsing image, and completion | finish. The figure which shows the other example of application of this invention

The video surveillance system, video processing server, and surveillance video browsing / distribution method of the present invention will be described with reference to the drawings.
As shown in FIG. 1, the video monitoring system 10 is roughly composed of a plurality of cameras 20, an information processing server 30, and a monitoring monitor 40.
A plurality of cameras 20 are arranged to photograph a plurality of monitoring objects (in this embodiment, monitoring objects 1 to 4) moving in the monitoring area 50. Each camera 20 may be arranged so that the shooting ranges overlap, i.e., to shoot the same place in the surveillance area 50, or so that the shooting ranges are independent, i.e., the shooting ranges do not overlap. May be. In the case where the shooting ranges are arranged so as to overlap, it is preferable to adjust the placement location and the viewing angle (view angle) of each camera 20 so that the monitoring target can be shot from different directions.

The information processing server 30 transmits and receives various types of information such as video data to the camera 20 and the monitor 40. The installation location of the information processing server 30 is not particularly limited, and may be, for example, a facility where the camera 20 is arranged, a monitoring room where the monitoring monitor 40 is arranged, or a location away from the facility and the monitoring room.
The information processing server 30 includes a communication unit 31 that transmits and receives various types of information to and from the camera 20 and the monitoring monitor 40, a database 32 that stores video data captured by the camera 20, and performs various processes on the video data. The image processing means 33 to perform is provided.
Various types of information can be transmitted and received via the communication unit 31 via an information communication network such as the Internet. The information processing server 30 is connected to the camera 20 and the monitor 40 by a connection cable or by a wireless LAN. A known communication method such as wireless communication can be used. Various connection types such as line connection and packet connection are possible. Also, well-known methods can be used for the output format and signal processing method of various information.

  The video processing means 33 is composed of a CPU, a RAM, and a ROM. The CPU reads out various programs stored in the ROM and each video data stored in the database 32, and executes the information processing server 30 by appropriately executing them. Take overall control. As will be described in detail later, the video processing means 33 analyzes the video data transmitted from each camera 20, and creates an optimal browsing video that summarizes the progress before and after the occurrence of the dangerous event accurately and simply. The created optimum browsing video is transmitted to the monitoring monitor 40 via the communication unit 31.

The monitor 40 is provided for receiving and displaying the optimum browsing video transmitted from the video processing means 33 via the communication unit 41. The monitoring monitor 40 corresponds to a stationary monitor arranged in a monitoring room or a monitor of a portable information terminal possessed by a monitoring person.
When the monitor wants to monitor a monitoring target different from the monitoring target displayed in the optimal browsing video being browsed, the monitoring person may select the new monitoring target via the input means. In this case, information indicating that a new monitoring target has been selected is transmitted to the information processing server 30 via the communication unit 41, and a new optimum browsing video is created by the video processing means 33.

Next, a video analysis process executed by the video processing means 33, that is, a monitoring video browsing / delivery method will be described.
2 to 4 show an outline of the processing, and FIG. 2 shows a flow until the video processing means 33 generates a continuous motion trajectory of each monitoring object based on the video taken by each camera 20. .
First, each camera 20 captures an image of a monitoring target moving in the monitoring area 50 (see FIG. 3A), and each video data is stored (input) in the database 32 of the information processing server 30 (step S10).
The image processing means 33 calls an appropriate program stored in the ROM, separates the foreground and the background for each image data (step S11a, see FIG. 3 (b)) and the geometrical structure of the camera coordinate system. Calibration is performed (in particular, it refers to unifying the coordinates of independent camera coordinate systems into one world coordinate system) (step S11b).

Next, the video processing means 33 integrates the information on the foreground separated from each video data (see step S12, FIG. 3C), and calculates the position of each monitoring target in the monitoring area 50 in time series ( Step S13, FIG. 3 (d)). Then, how each monitoring object moves in the monitoring area 50, that is, a continuous motion trajectory of each monitoring object is generated (see step S14, FIG. 3 (e)).
The graph of FIG. 5 schematically shows the movement trajectory of each monitoring target, and it can be seen how each monitoring target moves in the monitoring area 50 while approaching or moving away.

FIG. 4 shows a flow until the optimum browsing video is generated based on the continuous motion trajectory of each monitoring target.
The video processing means 33 that has generated the continuous motion trajectory (step S20), then detects the group interaction (step S21). A group interaction is defined as a continuous and stable spatial approach that occurs between a plurality of monitored objects within a certain period of time. Objects to be monitored such as people, animals, vehicles, etc., and objects that are present in the surveillance area 50 such as bags, suitcases, fire extinguishers, telephone boxes, etc. and cannot be moved by themselves (hereinafter referred to as monitoring objects) , “Specific environmental objects”).

FIG. 6 shows a flowchart of an example of a group interaction detection method, and FIG. 7 shows an image diagram.
The video processing means 33 generates one continuous motion trajectory for each monitoring target (see the leftmost diagram in FIG. 7A. The circle represents the monitoring target, and the arrow line represents the motion trajectory and its direction). The strength of the group interaction between the monitoring target and another monitoring target is determined (steps S30 and S31).
If the strength of the group interaction is strong (high), it is likely that both or one of the two monitored objects are approaching with some interest, so there is also the possibility of a dangerous event occurring there. It can be estimated that it is expensive.
Specifically, the continuous motion trajectory is composed of a set of a plurality of frames arranged continuously along the time series (see FIG. 7B), and the video processing means 33 monitors each monitor in the frame at time t. Calculate the strength of the group interaction between the subjects (here 6 monitoring subjects as an example).

  There are a plurality of methods for calculating the strength of the group interaction. For example, it may be determined simply by the distance between two monitoring targets. In this case, when the distance between the two monitoring targets is short, the strength of the group interaction becomes high. The center table in FIG. 7A shows the strength of the group interaction regarding the monitoring objects 1 to 6.

Further, as another example of the method for calculating the strength of the group interaction, a method in which the position of the monitoring target and the moving speed are considered will be described.
First, the following observations regarding the behavior of pedestrians are known as previous research results.
・ Pedestrians tend to maintain exercise at the most comfortable speed personally (the most comfortable speed is 1.34 m / s, with a standard deviation of 0.26 m / s)
・ Prefers linear motion ・ Private field exists to avoid collision ・ More space is needed in front of the direction of movement ・ Closer than repulsive group interaction to avoid The group interaction of the attracting effect to do is far more affected-both the repulsive effect and the attracting effect depend on the speed

  Based on this observation result, the intensity distribution of the group interaction can be defined by the positional relationship and velocity relationship between individuals (between monitoring targets). That is, the intensity distribution of the group interaction is defined as a function depending on the speed (see FIG. 8).

また、正面強度は次式で表される。

数２中、σ^Vは有効作用範囲と呼ばれ、距離上で強度の分布を制御するパラメータである。また、σ^θは角度上で強度の分布を制御するパラメータであり、次式で表される。

上式により、個体Aの速度が高ければ高いほど、群相互作用の目標個体が運動方向上に存在する確率が高くなり、群相互作用の強度も大きくなる。 When an individual A at position x _A moves at a speed v _A and an individual B at position x _B moves at a speed v _B , the strength with which the individual A performs group interaction with the individual B is It is expressed by the following equation consisting of a front strength representing separation and a side strength representing avoidance.

The front strength is expressed by the following formula.

In Equation 2, σ ^V is called an effective working range, and is a parameter that controls the intensity distribution over distance. Σ ^θ is a parameter for controlling the intensity distribution on the angle, and is expressed by the following equation.

From the above equation, the higher the speed of the individual A, the higher the probability that the target individual of the group interaction exists in the movement direction, and the strength of the group interaction also increases.

速度v_Aがゼロに近ければ近いほど、α_A(x_B)が１に近づき、前後方の区別がなくなる。また、速度v_Aが増大するとともに、α_A(x_B)がパラメータβに減衰し、群相互作用の強度が運動方向の前方に集中していく。 Assuming that an individual is more likely to have group interaction with a forward target in the direction of motion than that of a backward target and group interaction, the distinction is expressed by the function α _A (x _B ) .

The closer the velocity v _A is to zero, the closer α _A (x _B ) is to 1, and there is no distinction between front and rear. Further, as the velocity v _A increases, α _A (x _B ) attenuates to the parameter β, and the strength of the group interaction is concentrated forward in the movement direction.

ここでω^Vとω^θは二つの制御パラメータである。上述の群相互作用の強度による両個体間の典型的な群相互作用の表現方法を図９に示す。
以上のように、監視対象の位置及び移動速度を考慮して群相互作用の強度を算出してもよい。 The side strength represents a private field, is effective only within a short range, and is modeled as not sensitive in angle, and is expressed as the following equation.

Here, ω ^V and ω ^θ are two control parameters. FIG. 9 shows a typical method for expressing group interaction between both individuals based on the strength of the group interaction described above.
As described above, the strength of the group interaction may be calculated in consideration of the position of the monitoring target and the moving speed.

Next, the image processing means 33 classifies the intensity of group interaction based on a certain threshold value, thereby classifying two or more monitoring targets having strong group interaction as one individual group.
The rightmost diagram in FIG. 7A shows a state of being divided into an upper individual group including four monitoring targets and a lower individual group including two monitoring targets.
As shown in FIG. 7 (b), the video processing means 33 classifies the individual group performed in the frame at time t for all frames from the start of shooting at t = 0 to the end of shooting (step S32). Next, an operation for connecting the populations on the time axis is performed (step S33). The individuals connected in this way can be regarded as so-called hot spots where there is a high possibility that a dangerous event has occurred within that range (see FIG. 10).
In FIG. 10, when a fire occurs, one person who is a monitoring target approaches a fire extinguishing device (a specific environmental object) that is also a monitoring target, and a state where an individual group 1 is formed, and two persons who are monitoring targets The other population 2 is formed.

  FIG. 7 (c) shows that as a result of performing the classification of the population over the entire imaging time, it may temporarily belong to one of the individual groups, but immediately leaves the individual group (the group interaction) Therefore, the monitoring target that can be judged to be less likely to be involved in the dangerous event is excluded from the population (removed as noise). By performing such noise removal work, it is possible to extract only those individuals whose group interaction strength is high and stable for a certain period of time. Can be increased. However, such noise removal work is not essential for generating the optimal browsing video.

  FIG. 11 is a graph in which individual groups represented by ellipses are superimposed on the graph of FIG. 5, and the numbers in the ellipses represent the numbers of the respective monitoring targets. For example, when the person of the monitoring target 3 first approaches the monitoring targets 1 and 2, an individual group composed of the monitoring targets 1 to 3 is formed, and then the individual group is canceled by moving away from the individual group. When the monitoring target 3 next approaches the monitoring target 4, an individual group composed of the monitoring targets 3 and 4 is formed, and thereafter, the individual group is canceled by leaving from the individual group. Then, it can be seen that the monitoring object 3 again approaches the monitoring object 2 to form an individual group composed of the monitoring objects 2 and 3, and then is eliminated.

Next, the video processing means 33 generates a transition relation diagram between the individual groups (step S22 in FIG. 4).
FIG. 12 shows a flowchart when the video processing means 33 generates a transition relation diagram between individual groups.
First, the video processing means 33 detects the time when each individual group was formed and the time when it was resolved, and performs numbering with the imaging start time as 1 for each time (steps S40 and S41).
FIG. 13 shows an image of the numbered state. The right-pointing arrow lines in FIG. 13 correspond to the monitoring targets 1 to 4 in order from the top, and the solid line portion in each line indicates the time zone in which the corresponding monitoring target forms an individual group, and the alternate long and short dash line portion Indicates a time zone in which a population is not formed, that is, a time zone in which the monitoring target is acting alone.
For example, the imaging start time is numbered 1 in circles in FIG. 13, and then the number 2 at the time when the monitoring object 1 and 4 approach each other and the individual group is formed, the time when the individual group is canceled Is numbered 3. In the time zone numbered 2 to 3, the lines of the monitoring targets 1 and 4 forming the population are represented by solid lines, and the lines of the monitoring targets 2 and 3 not forming the population are represented by the alternate long and short dash lines ing.

Next, the video processing means 33 creates a graph with each numbered time as a node and a line connecting these nodes as a link.
Specifically, as shown in FIG. 14A, by connecting nodes corresponding to each time of formation and cancellation of the individual group, a link corresponding to the time zone in which the individual group is formed (for example, a number) Create a link between nodes 2 and 3 or a link between nodes 7 and 9. In addition, links (such as links connected by nodes numbered 1 and 4 or nodes numbered 9 and 10) that correspond to the time zone in which the population is not formed (time zone in which each monitoring target acts independently) Link etc.) is also created (step S42).

  In addition, for the link corresponding to the time zone in which the individual group is formed, the cost of transition from one individual group to another is calculated based on the strength of the group interaction, and this is added as a weight. is there. Further, the cost is also calculated for the link corresponding to the time zone when the individual group is not formed, and this is added as a weight (step S43). For example, when a number is expressed in parentheses on the link in FIG. 14A (for example, (1,2,3)), the weight related to the individual group that is the monitoring target of the number (number) is given to the link. If a single number is shown on the link, or if multiple numbers are shown with a slash (for example, 4 or 1/2/3), that number This indicates that the monitoring target is acting alone and the weight related to this is attached to the link.

m番目のノードv_mに対応する時刻はt_mとする。ノードv_m1とノードv_m2を結ぶリンクe_nは群相互作用g_nに対応する。群相互作用g_nは当該群相互作用に含まれる個体の番号の集合として次式で定義される。

A specific method for calculating the cost (weight) is as follows.
First, it is assumed that the transition graph G includes a set V of M nodes and a set E of N links represented by the following equations.

The time corresponding to the _mth node v _m is t _m . Link e _n connecting node v _m1 and node v _{m @ 2} corresponds to the group interaction g _n. The group interaction g _n is defined by the following equation as a set of individual numbers included in the group interaction.

群相互作用g_nの時刻tでの重要度は成員個体の重要度の総和として次式で定義される。

また、リンクe_nのコスト（重み）は対応期間における群相互作用g_nの総重要度の負数として次式で定義される。

以上で映像処理手段33による各個体群間の遷移関係図の作成作業が完了する（ステップS44及び図４のステップS22）。 In order to formulate uniformly, a time zone in which no individual group is formed is regarded as a special individual group composed of a single individual, and this is recognized as a group interaction. It is assumed that the speed of the individual o _i at time t is v _it and position x _it . The importance of individual o _i at time t is defined as the sum of the strengths of group interactions from other individuals as follows:

The importance of group interaction g _n at time t is defined as the sum of the importance of member individuals by the following equation.

Moreover, the cost (weight) of the link e _n is defined by the following equation as a negative number of the total importance of the group interaction g _n in the corresponding period.

Thus, the creation of the transition relationship diagram between the individual groups by the video processing means 33 is completed (step S44 and step S22 in FIG. 4).

The generation of the optimum browsing content (step S23) is started when the monitor designates a specific monitoring target using the input means.
Specifically, when a specific monitoring target (for example, monitoring target 3) is designated by the monitor, the video processing means 33 uses the above transition relationship diagram to indicate a route that minimizes the total cost of transition for the monitoring target 3. Is calculated. In this case, by providing a rule that the observer can always see only one screen, even if there are multiple links from one node, only the link with the lowest total cost of transition can be selected. become.

Based on such rules, the video processing means 33 calculates a route (route indicated by a thick line arrow in FIG. 14A) that minimizes the total cost of transition, and edits the video constituted by the route. To create the optimal viewing video. FIG. 14B is an extracted route from the graph of FIG. 14A where the total transition cost for the monitoring target 3 is minimized.
The created optimum browsing video (monitoring video) is output (distributed) to the monitoring monitor 40 via the communication unit 41 (step S24), and the monitoring person checks (monitors) this.

The video processing means 33 is preferably processed and output so as to have a resolution corresponding to the performance of the monitor 40. Further, the optimal browsing video may be so-called forward playback that is played back from the shooting start time point, or so-called reverse playback that is played back from the shooting end time point.
The optimal viewing video created in this way is a summary of the pre- and post-hazardous events that occur accurately and simply, reducing the burden of monitoring work by the surveillance staff in both real-time monitoring and subsequent viewing. And improve work efficiency.

When the total shooting time is long, the video processing means 33 can change the playback speed of the optimal browsing video based on the strength of the group interaction.
As shown in FIG. 15, the regeneration speed is 1 × (1 ×), 1.5 × (1.5 ×), and 2 × for a route including a population having a strong group interaction, that is, a route that is likely to be related to a dangerous event. (2x) The speed of playback is reduced to a speed that is easy for the observer to check, and in the route where the group interaction is weak or the surveillance target is acting alone, i.e., the route that is not likely to be related to the dangerous event For example, if the speed is increased to such an extent that it cannot be visually confirmed from 10 times to infinity, the part is substantially skipped, and further reduction of the burden of monitoring work and improvement of efficiency can be realized. In addition to the strength of the group interaction described above, the regenerative speed adjustment criteria may be based on the length of time from when a single population is formed until it is resolved, or manually adjusted by a supervisor. Also good.

In addition, the monitoring person who is checking the optimum browsing image can specify a monitoring target or an individual group that is different from the initially specified monitoring target via the input unit.
In this case, the video processing means 33 recalculates a route having the minimum transition total cost among the routes starting from the end point of the node including the monitoring target or the individual group on the transition relation diagram and starting from the photographing end point. This is output as an optimal browsing video. FIG. 16 (b) shows that the video processing means 33 recalculates when the monitor changes the monitoring target from 3 to 2 in the node of number 10 while checking the optimal browsing video by reverse playback. This shows a state changed from the original route (FIG. 16A).

  FIG. 17 shows an example of an image layout displayed on the monitoring monitor 40, and the video of each camera 20 is shown in the top row. On the left side below it is the video represented by a 2D or 3D bar that color-codes the monitoring target or population, and on the right side is the optimal viewing video. Below that, as shown in FIG. 13, various information such as a graph showing a time zone in which each monitoring target forms an individual group and a time zone in which the individual is acting alone and a playback speed are shown. In the lower row, various information such as buttons for playing, stopping, fast-forwarding, rewinding, and the like of the video and playback time are shown. FIG. 18 shows an actually created image layout.

FIG. 19 is a flowchart showing processing operations performed by the video processing means 33 from the start to the end of the optimal browsing image check operation by the monitor.
First, when the monitor designates the monitoring target or the individual group via the input means, that is, when the viewing condition is designated (step S50), the video processing means 33 generates a transition relation diagram between the individual groups (step S51), A route that minimizes the total transition cost is determined (step S52).
Next, the video processing means 33 determines the playback speed (step S53), selects the video data most suitable for use in the optimal browsing image from the video data of each camera 20, and determines the screen range, that is, A part of the video is enlarged / reduced and displayed as necessary (step S54), and after appropriate image processing, it is output to the monitoring monitor 40 as an optimal browsing image (step S55).

When the video processing means 33 outputs the optimum browsing video to the end, the reproduction is finished (YES in step S56), and the check work by the monitoring person is finished.
On the other hand, if the optimal browsing video has not been output to the end (NO in step S56), the optimal browsing image is continuously output until an instruction to change the browsing conditions is received from the monitoring staff (NO in step S57).
If there is an instruction to change viewing conditions from the monitoring person (YES in step S57), the type of the instruction, that is, “designation of video resolution” (step S58), “designation of playback speed” (step S59), “ Reprocessing is appropriately performed in accordance with “specifying individual group” (step S60) and “designating monitoring target” (step S61), and the optimal browsing image is generated again.

Another application example of the present invention will be described with reference to FIG.
FIG. 20 (a) shows a video showing the behavior of each monitoring target (person 1 to 3) from when a fire occurs to the start of fire fighting using multiple cameras (cameras 1 to 4) over time. FIG. 20B shows the presence / absence of the individual group formation, the person included in the individual group, and the camera video used for the optimum browsing image.

First, at time 1, camera 3 captures the occurrence of a fire, but each monitoring target has not yet recognized the occurrence of a fire.
At time 2, people 2 and 3 discover a fire, and a population is formed between them. At time 3, the person 2 informs the person who is in the vicinity of the fire extinguisher that the fire has occurred, and a population is formed between them. On the other hand, the person 3 moves in the direction of the fire extinguisher. At time 4, persons 1 and 3 with a fire extinguishing device enter the fire fighting operation, and a population is formed at this time. On the other hand, the person 2 moves in a direction outside the monitoring area 50 to report that a fire has occurred.
When such a video is shot, the optimal viewing video is the video of camera 2 at time 1, the video of camera 3 shooting the state where the population was formed at time 2, and the new population at time 3 The video of the camera 3 in which a new individual group is formed at time 4 is selected.

  Monitoring work for both real-time monitoring and subsequent viewing by creating an optimal viewing video that accurately and simply summarizes the progress of a dangerous event before and after from a large amount of video data captured by multiple cameras. The present invention relates to a video monitoring system, a video processing server, and a monitoring video browsing / distribution method that reduce the burden on the user and increase work efficiency, and has industrial applicability.

10 Video surveillance system
20 Camera
30 Information processing server
31 Communication Department
33 Video processing means
40 Monitor
41 Communication Department
42 Input means
50 surveillance area

Claims (12)

A plurality of cameras for photographing a plurality of monitoring targets moving in the surveillance area;
Video processing means for processing each video captured by the plurality of cameras;
A monitoring monitor for displaying the video processed by the video processing means,
The video processing means calculates the strength of the group interaction between the monitoring targets that changes over time based on the trajectory information of each monitoring target, and there are two or more monitoring targets that have a strong group interaction. In some cases, these monitoring targets are classified as one individual group, and then the cost of transitioning to each individual group is defined based on the strength of the group interaction using graph theory. A video monitoring system, wherein the configured video is output to the monitoring monitor as an optimal browsing video.   When the monitor selects an arbitrary population or monitoring target, the video processing means recalculates a route with the lowest total cost among the routes including the individual population or the monitoring target, and outputs it as an optimal viewing video The video surveillance system according to claim 1, wherein:   The video monitoring system according to claim 1, wherein the video processing unit changes the playback speed of the optimal browsing video based on the strength of group interaction.   The video monitoring system according to any one of claims 1 to 3, wherein the video processing means processes and outputs the optimum browsing video so as to have a resolution corresponding to the performance of the monitoring monitor. A plurality of monitoring objects moving in the monitoring area are photographed with a plurality of cameras, and each photographed image is processed and output to a monitoring monitor, and then provided with video processing means.
The video processing means calculates the strength of the group interaction between the monitoring targets that changes over time based on the trajectory information of each monitoring target, and there are two or more monitoring targets that have a strong group interaction. In some cases, these monitoring targets are classified as one individual group, and then the cost of transitioning to each individual group is defined based on the strength of the group interaction using graph theory. A video processing server characterized in that a configured video is output to the monitoring monitor as an optimal browsing video.   When the monitor selects an arbitrary population or monitoring target, the video processing means recalculates the route with the lowest total cost among the routes including the individual population or monitoring target, and outputs it as an optimal viewing video The video processing server according to claim 5, wherein:   The video processing server according to claim 5 or 6, wherein the video processing means changes the playback speed of the optimal browsing video based on the strength of group interaction.   The video processing server according to any one of claims 5 to 7, wherein the video processing means processes and outputs the optimum browsing video so as to have a resolution corresponding to the performance of the monitor. Photographing a plurality of monitoring objects moving in the monitoring area with a plurality of cameras;
Processing each video taken by the plurality of cameras;
Displaying the processed video on a surveillance monitor,
Further, based on the trajectory information of each monitoring target, a step of calculating the strength of the group interaction between the monitoring targets that changes over time, and when there are two or more monitoring targets that have strong group interaction Consists of a step to classify these monitoring targets as a single population, a step to define the cost of transitioning to each population using graph theory based on the strength of the group interaction, and a path that minimizes the total cost A method for browsing / distributing surveillance video, comprising the step of outputting the video to be monitored as the optimum viewing video to the monitoring monitor.   Provided with a step of recalculating a route with the lowest total cost among routes including the individual group or the monitoring target when the monitor selects an arbitrary population or the monitoring target, and outputting it as an optimal viewing video The method for browsing and distributing surveillance video according to claim 9.   The monitoring video browsing / delivery method according to claim 9 or 10, further comprising a step of changing a playback speed of the optimal browsing video based on the strength of the group interaction. The method for browsing and distributing a surveillance video according to any one of claims 9 to 11, further comprising a step of processing and outputting the optimum browsing video so as to have a resolution corresponding to the performance of the monitoring monitor.

Images