JP2004056473A - Monitoring controller - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a monitoring controller controlling the frame rate of image recording from present monitoring video. <P>SOLUTION: The monitoring controller 101 is provided with a neural network (NW) 111 outputting recognition information corresponding to an image photographed by a camera 102 based on a learning result, a control means 108 performing control based on recognition information, a short time storage means 106 which temporarily preserves image data and a storage means 109 recording image data. NW learns a relation between the image and the degree of an emergency of a phenomenon displayed by the image and recognizes the degree of emergency corresponding to the image of the camera. The control means controls a frame rate of image data recorded in the storage means by recognition information of NW. When a high-order emergency such as invasion of a suspicious person occurs, the frame rate is made high and the high quality camera image is kept. Then, the frame rate is made slightly lower in a emergency such as vibration of glass due to strong wind and the camera image of somewhat high image quality is kept. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a surveillance system for monitoring a predetermined place or area using a surveillance camera, and a surveillance control device, and in particular, controls a frame rate of image recording, a camera control, and a monitor based on a current camera image. This enables control and the like.
[0002]
[Prior art]
Conventionally, in a surveillance system, an image of a surveillance camera is displayed on a monitor, and this image is recorded on a VTR, a hard disk, or the like.
Japanese Patent Application Laid-Open No. 2000-253390 describes a monitoring and recording device that controls recording of video of a monitoring camera according to whether or not an abnormal situation has occurred at a monitoring location. This apparatus is provided with an image recognizing means for recognizing an image of the surveillance camera. In a normal time when no abnormality occurs, the image of the surveillance camera is recorded at a low frame rate and a low image quality. Is detected, the frame rate is increased, and the image of the surveillance camera is recorded with high image quality.
[0003]
[Problems to be solved by the invention]
However, in this conventional monitoring and recording apparatus, the recording frame rate in normal times and in the event of an emergency is set in advance, and image data can be stored only at that frame rate.
In practice, there are various levels and types of emergencies, for example, from higher-level emergencies, such as when a suspicious person enters, to lower-level situations, such as when glass is vibrating in a strong wind. Exist up to. The surveillance system uses a higher frame rate to preserve higher quality images when a suspicious person enters, while lowering the frame rate slightly when the wind is vibrating the glass. Therefore, it is required to control image recording according to the degree of an emergency, such as storing an image of a certain image quality.
[0004]
Also, in a surveillance system, if the glass is vibrating due to strong wind, the camera may be fixed and shooting can be continued, but if a suspicious person invades, camera control that tracks the suspicious person with a camera is necessary. Become.
In addition, it is necessary to control the monitor selection such that an image when a suspicious person enters is sent to a monitor in the security department, and an image when glass is vibrating due to strong wind is sent to a monitor in the building management department.
[0005]
The present invention is to solve such a conventional problem, and provides a monitoring control device that can control a frame rate of image recording from a current monitoring video, and can perform camera control and monitor control, It is another object of the present invention to provide a monitoring system using the monitoring control device.
[0006]
[Means for Solving the Problems]
Therefore, in the present invention, a neural network that outputs recognition information corresponding to an image photographed by a photographing unit based on a learning result to a monitoring control device, and control is performed based on recognition information output from the neural network. A control unit, a short-term storage unit for temporarily storing image data of an image photographed by the photographing unit, and a storage unit for recording image data transferred from the short-term storage unit; And learning the relationship between the degree of urgency of the event represented by the image and the degree of urgency corresponding to the image photographed by the photographing means, and the control means recognizes the degree of urgency based on the recognition information output from the neural network. Based on this, the frame rate of the image data transferred from the short-term storage unit to the storage unit is controlled.
With this configuration, when a higher-level emergency such as intrusion of a suspicious person occurs, the frame rate is increased, the camera image is stored with higher image quality, and lower-level emergencies such as vibration of glass due to strong wind When a situation occurs, the camera image is saved with a certain image quality by lowering the frame rate somewhat.
[0007]
Further, the monitoring control device includes a neural network that outputs recognition information corresponding to an image captured by the imaging unit based on a learning result, and a control unit that performs control based on the recognition information output from the neural network. The neural network learns the relationship between the learning image and the control contents of the photographing means, and recognizes the control contents of the photographing means corresponding to the image photographed by the photographing means. The photographing means is controlled on the basis of.
With this configuration, even if the user does not control the camera from the controller, for example, if a suspicious person is reflected in the current camera image, camera control for tracking the suspicious person is automatically performed.
[0008]
Further, the monitoring control device includes a neural network that outputs recognition information corresponding to an image captured by the imaging unit based on a learning result, and a control unit that performs control based on the recognition information output from the neural network. The neural network learns the relationship between the learning image and the display means for displaying the image, recognizes the display means corresponding to the image photographed by the photographing means, and controls the control means based on the recognition information. Thus, a display means for displaying an image photographed by the photographing means is selected.
According to this configuration, a monitor suitable for displaying the image is selected from the current camera image without transmitting the image data from the current camera image to the monitor without the user selecting the monitor from the controller.
[0009]
Further, in the present invention, a monitoring control device is configured by combining the above-described configuration for performing camera control and the configuration for performing monitor selection control, and further includes a configuration for controlling the frame rate of a recorded image described above. The monitoring control device is configured in combination.
In this combined monitoring and control device, each operation of a plurality of combined configurations is possible.
[0010]
Further, in the present invention, a monitoring system is configured by the monitoring control device, a photographing unit for photographing the monitoring area, and a display unit for displaying an image photographed by the photographing unit.
In this monitoring system, the processing of image recording, camera control, and monitor control can be automatically switched according to the current camera image captured by the image capturing means by the operation of the monitoring control device.
[0011]
BEST MODE FOR CARRYING OUT THE INVENTION
(1st Embodiment)
In the first embodiment of the present invention, a monitoring control device of a monitoring system that controls recording of monitoring image data according to the degree of an emergency will be described.
As shown in FIG. 1, the monitoring system captures a moving image and outputs analog image data, a camera 102, a monitor 103 that displays a moving image, and records monitoring image data captured by the camera 102. A monitoring control device 101 for outputting to the monitor 103; an A / D converter 104 for converting a data format (analog data) handled by the camera 102 to a data format (digital data) handled in the monitoring control device 101; And a D / A converter 105 for converting a data format (digital data) handled by the monitor 103 into a data format (analog data) handled by the monitor 103.
[0012]
The monitoring control device 101 temporarily stores the image data converted by the D / A converter 105 together with the time index, and inputs the stored data from the short-term storage unit 106 that outputs the stored data at a specific frame rate, and the camera 102. An information recognizing unit 107 for extracting a feature of the image data to perform a recognition operation, a control circuit 108 for determining a frame rate, controlling a controller (not shown) operated by a user, calculating, transmitting / receiving, and the like; A recording device 109 comprising a hard disk, a digital video disk, or the like, for recording data, information data, etc., and an information recognizing unit 107, which is provided with a feature of information having a specific temporal and spatial structure from input information. Is extracted and recognized, and a neural network 111 is extracted so that the neural network 110 can easily recognize the neural network. A pre-processing unit 110 that performs, for example, a process of emphasizing or suppressing a specific spectrum on image data input to the neural network 111, and a learning that saves a camera image of an emergency that may have occurred in the monitoring control device 101 up to now. A standard learning pattern storage unit 112 for storing patterns of image data for use and patterns of teacher signals indicating the degree and frequency of an emergency in the stored image data for learning.
[0013]
Next, the operation of the monitoring control device 101 will be described.
First, it is necessary to make the neural network 111 learn about the degree of emergency before performing the control operation of the monitoring control device 101. Learning of the neural network 111 is performed using a pattern group of learning image data stored in the standard learning pattern storage unit 112 and a pattern group of a teacher signal. In FIG. 1, the flow of data in this learning is indicated by double-line arrows.
[0014]
The learning image data is preprocessed by the preprocessing unit 110 and input to the neural network 111. If the pattern of the learning image data is a high-order emergency, a teacher signal pattern that increases the frame rate is input from the standard learning pattern storage unit 112 to the neural network 111, and the pattern of the learning image data is Is a lower-order emergency, a teacher signal pattern whose frame rate is slightly lower than that of the higher-order emergency is input to the neural network 111, and the learning image data is If it is a pattern, a teacher signal pattern such that the frame rate becomes a lower value than that of a low-order emergency is input to the neural network 111, and a corresponding teacher signal is obtained corresponding to each learning image data. The learning is repeated by changing the connection weight of the neural network 111 so that the learning can be performed.
[0015]
When the camera 102 outputs a captured image (moving image), the A / D converter 104 A / D converts the image into an image data format (digital data) handled by the monitoring control device 101. The converted image data is temporarily stored in the short-term storage unit 106 at a high frame rate of the output image of the camera 102 together with the time index.
[0016]
FIG. 2 shows an example of the image data and the time index stored in the short-term storage unit 106. Here, the image data is stored in the short-term storage area, and the time index is stored as a set of the date and the address number of the short-term storage area.
The A / D-converted image data is also input to the pre-processing unit 110, and the pre-processing unit 110 applies, for example, a specific spectrum to the image data so that the neural network 111 has a high recognition rate. For example, a process of emphasizing or suppressing or flattening the spectral power is performed. The image data processed by the preprocessing unit 110 is input to the neural network 111.
The trained neural network 111 extracts specific temporal and spatial structure features from the input information, and recognizes the degree of emergency from the contents already learned.
[0017]
The control circuit 108 determines the frame rate in accordance with the degree of emergency recognized by the learned neural network 111, specifies the time index and the frame rate for specifying the image data in the short-term storage unit 106, and stores the frame rate. A control command for instructing storage of the image data in the device 109 is sent. The short-term storage unit 106 stores the specified image data in the storage device 109 at the specified frame rate according to the control command. Further, the image data stored in the recording device 109 may be subjected to a process of compressing at the time of recording and expanding at the time of reading.
[0018]
As described above, the supervisory control device according to the first embodiment uses the neural network to recognize the level of the generated emergency, and stores the image at a frame rate according to the level.
As a result, in the event of a higher-order emergency such as intrusion of a suspicious person, the frame rate is increased, the moving image is stored with higher image quality, and in the event of a lower-order emergency such as vibration of glass due to strong wind, the frame rate is increased. And the moving image is saved with a certain image quality.
[0019]
(Second embodiment)
The monitoring control apparatus according to the second embodiment of the present invention stores image data input from a camera during operation, and performs relearning using this image.
As shown in FIG. 3, the monitoring control device includes an information recognizing unit 307 that, in addition to the preprocessing unit 310, the neural network 312, and the standard learning pattern storage unit 314, inputs the input to the preprocessing unit 310 to the standard learning pattern An input conversion switch 311 for switching to the storage unit 314 or the A / D converter 304 and an output conversion switch 313 for switching the connection destination of the neural network 312 to the standard learning pattern storage unit 314 or the control circuit 308 are provided. The image data input from the A / D converter 304 to the pre-processing unit 310 is also stored in the standard learning pattern storage unit 314. Other configurations are the same as those of the first embodiment (FIG. 1).
[0020]
Next, the operation of the monitoring control device 301 will be described.
First, before the operation of the monitoring control device 301, the input conversion switch 311 and the output conversion switch 313 are connected to the standard learning pattern storage unit 314, and the degree of emergency by the neural network 312 is determined in the same manner as in the first embodiment. Learning is performed.
After the initial learning is completed, the operation of the monitoring control device 301 is started. After the operation starts, a normal monitoring operation of monitoring with an image of the camera 302 and a re-learning operation of learning again based on the monitoring contents of the normal monitoring operation. Execute
[0021]
In the case of the normal monitoring operation, the input changeover switch 311 is connected to the A / D converter 304, and the output changeover switch 313 is connected to the control circuit 308, and the same operation as in the first embodiment is performed. At this time, the image data input to the preprocessing unit 310 is stored in the standard learning pattern storage unit 314.
In the case of the re-learning operation, the input changeover switch 311 and the output changeover switch 313 are connected to the standard learning pattern storage 314, and the neural network 312 is re-learned. In the re-learning, the image data pattern stored in the standard learning pattern storage unit 314 during the normal monitoring operation is input to the neural network 312 via the preprocessing unit 310, and the image data pattern is used as a teacher signal at this time. When a large amount of data is stored in the standard learning pattern storage unit 314 during the normal monitoring operation, a teacher signal for increasing the frame rate is input to the neural network 312. Conversely, during the normal monitoring operation, the standard learning pattern storage unit 314 is input. If the number stored in is small, a teacher signal for lowering the frame rate is input to the neural network 312 to perform learning.
The timing of switching between the normal monitoring operation and the re-learning operation is such that if the frequency of occurrence of an emergency detected by the camera 302 is high, the frequency of switching is increased, and frequent re-learning is performed. If it is low, the frequency of switching is reduced.
[0022]
As described above, the surveillance control device according to the second embodiment of the present invention executes relearning using information input from an operating camera in addition to the normal monitoring operation described in the first embodiment. For this reason, in the first embodiment, the frame rate can be determined only by learning before operation. However, in this monitoring control device, an emergency situation that is likely to occur at the place where the camera is placed during operation is learned. It can be set so that moving images are recorded at a higher frame rate for those with a high frequency of emergency situations, and at a lower frame rate for an emergency situation that does not occur at all. That is, the frame rate for recording the emergency can be set autonomously in accordance with the tendency of the emergency occurring in the place being monitored.
[0023]
(Third embodiment)
The monitoring control device according to the third embodiment of the present invention automatically controls the movement of the camera based on the current camera image.
As shown in FIG. 4, the monitoring system generates a camera 402 for capturing a moving image, an A / D converter 403 for converting image data captured by the camera 402 into digital data, and a control command for the camera 402. A neural network 405 that recognizes what kind of camera control is to be performed from the input image data based on the learning result, and an image data input to the neural network 405. A pre-processing unit 404 that performs pre-processing of the above, a pattern group of learning image data in which the characteristics of the past camera image input to the monitoring control device 401 are stored, and a pattern group of the teacher signal corresponding to the learning image data Based on the stored standard learning pattern storage unit 407 and the recognition result of the neural network 405, the camera 4 It generates a second control command and a control circuit 406 to be transmitted to the camera 402.
[0024]
The control command of the camera 402 specifies the position and orientation of the camera, zoom, light amount, focus, and the like. The standard learning pattern storage unit 407 stores, as a teacher signal, the position and orientation of the camera corresponding to the learning image data. , Signals for setting zoom, light amount, focus, etc. are stored.
[0025]
Next, the operation of the monitoring control device 401 will be described.
First, it is necessary to make the neural network 405 learn what kind of camera control is to be performed based on the image of the camera 402 before the operation of the monitoring control device 401.
In this learning, a learning image data pattern storing characteristics of past camera images stored in the standard learning pattern storage unit 405 is input to the neural network 405, and when the learning image data pattern is input, The learning for changing the connection weight of the neural network 405 is repeatedly performed using such a teacher pattern for determining whether to issue a camera control command.
The operations of the camera 402, the A / D converter 403, and the preprocessing unit 404 are the same as in the first embodiment.
[0026]
When learning of the neural network 405 is completed and the current video of the camera 402 is input to the monitoring control device 401, the learned neural network 405 determines what kind of camera control is to be performed based on the image data processed by the preprocessing unit 404. Recognize The control circuit 406 generates a control command for the camera 402 based on the recognition result of the learned neural network 405 and transmits the control command to the camera 402.
[0027]
As described above, in the monitoring control apparatus according to the third embodiment of the present invention, since the neural network learns the control of the camera from the image of the camera, even if the user does not control the camera from the controller, the present control is performed. The direction and zoom of the camera can be automatically controlled so as to track the suspicious person from the camera image.
[0028]
(Fourth embodiment)
The monitoring control device according to the fourth embodiment of the present invention automatically controls video output to a monitor based on a current camera image.
As shown in FIG. 5, the monitoring system includes a camera 502 that captures a moving image, an A / D converter 504 that converts image data captured by the camera 502 into digital data, and a monitor 503 that displays a moving image. A D / A converter 505 that converts input digital data into analog data and outputs the analog data to a monitor 503, and a monitoring control device 501 that selects a monitor to which a camera image is to be output. A neural network 507 for determining which monitor 503 is to output a camera image based on a learning result, a preprocessing unit 506 for performing preprocessing of image data input to the neural network 405, and an input to the monitoring control device 501. Pattern group of learning image data which preserved the characteristics of the past camera image and its learning image data The standard learning pattern storage unit 509 and the group of patterns corresponding teacher signals are stored, and a control circuit 508 for transmitting a monitor selection control signal based on the recognition result of the neural network 507.
The standard learning pattern storage unit 509 stores, as a teacher signal, a signal specifying the monitor 503 to which the learning image data is to be output, corresponding to the learning image data.
[0029]
Next, the operation of the monitoring control device 501 will be described.
First, before the operation of the monitoring control device 501, it is necessary to make the neural network 507 learn what kind of monitor 503 outputs the camera image according to the image of the camera 502.
[0030]
In this learning, a learning image data pattern storing characteristics of past camera images stored in the standard learning pattern storage unit 509 is input to the neural network 507, and an output when the learning image data pattern is input is output. The learning weight is changed by changing the connection weight of the neural network 507 using the teacher signal pattern designating the destination monitor.
The operations of the camera 502, the A / D converter 504, and the preprocessing unit 506 are the same as those in the first embodiment.
[0031]
When learning of the neural network 507 is completed and the current image of the camera 502 is input to the monitoring control device 501, the learned neural network 507 recognizes which monitor should output the image data processed by the preprocessing unit 506. I do. The control circuit 508 selects the monitor 503 based on the recognition result of the learned neural network 507, and outputs a camera image to the monitor 503 via the D / A converter 503.
[0032]
As described above, in the monitoring control device according to the fourth embodiment of the present invention, since the neural network learns the monitor control for selecting the destination monitor of the camera image from the image of the camera, the user himself / herself uses the controller. Even without performing monitor control, it is possible to automatically select a monitor at the transmission destination according to the current camera image.
[0033]
Note that a plurality of configurations shown in each embodiment of the present invention can be combined. For example, by combining the third embodiment and the fourth embodiment, it is possible to configure a monitoring control device capable of performing both automatic control of a camera and automatic selection control of a monitor. In that case, it is necessary to have two, a neural network for camera automatic control and a neural network for monitor automatic control.
[0034]
Further, by combining the configurations of the first, third, and fourth embodiments, it is possible to configure a monitoring control device capable of automatic monitor control, automatic camera control, and recording control of the camera image. When a suspicious person invades the monitoring area, the camera is controlled to track the suspicious person, and the image is recorded in a storage device with a high frame rate for a high-order emergency and a high image quality, The image data is sent to a monitor that can be watched by a security officer.If the glass vibrates due to strong wind, etc., the camera position is fixed, and the image is displayed at a slightly lower frame rate for low-order emergency situations. In addition, it is possible to record the image data in the storage device with a certain image quality and transmit the image data to a monitor viewed by the building manager.
[0035]
【The invention's effect】
As apparent from the above description, the surveillance control device of the present invention can realize an advanced function of switching processing according to the current image of the surveillance camera by performing learning using a neural network.
[0036]
Regarding the recording of surveillance images, when a higher-level emergency such as intrusion of a suspicious person occurs, increase the frame rate, save higher-quality camera images, When an emergency occurs, the frame rate can be reduced slightly and camera images can be saved with a certain image quality.
[0037]
Regarding camera control, the operation of the camera can be controlled based on the current camera image, and when a suspicious person appears in the image, camera control for tracking the suspicious person can be automatically performed. .
Further, regarding the monitor control, a monitor for displaying the image can be selected and controlled from the current camera image.
In addition, a monitoring system using this monitoring control device can adaptively switch processing of image recording, camera control, and monitor control according to the current camera image by the operation of the monitoring control device.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of a monitoring control device according to a first embodiment of the present invention. FIG. 2 is a diagram illustrating data stored in a short-term storage unit of the monitoring control device according to the first embodiment of the present invention. FIG. 3 is a block diagram illustrating a configuration of a supervisory control device according to a second embodiment of the present invention; FIG. 4 is a block diagram illustrating a configuration of a supervisory control device according to a third embodiment of the present invention; Block diagram showing the configuration of the monitoring control device in the fourth embodiment of the present invention.
101 monitoring recording device 102 camera 103 monitor 104 A / D converter 105 D / A converter 106 short-term storage unit 107 information recognition unit 108 control circuit 109 recording device 110 preprocessing unit 111 neural network 112 standard learning pattern storage unit 301 monitoring record Device 302 Camera 303 Monitor 304 A / D converter 305 D / A converter 306 Short-term storage unit 307 Information recognition unit 308 Control circuit 309 Recording device 310 Preprocessing unit 311 Input conversion switch 312 Neural network 313 Output conversion switch 314 Standard learning pattern Storage unit 401 Camera control unit 402 Camera 403 A / D converter 404 Preprocessing unit 405 Neural network 406 Control circuit 407 Standard learning pattern storage unit 501 Monitor control unit 502 Camera 503 Monitor 504 A / D converter 505 D / A converter 506 Preprocessing unit 507 Neural network 508 Control circuit 509 Standard learning pattern storage unit

Claims (7)

A neural network that outputs recognition information corresponding to an image photographed by the photographing means based on a learning result;
Control means for performing control based on the recognition information output from the neural network,
A short-term storage unit for temporarily storing image data of an image captured by the imaging unit,
Storage means for recording the image data transferred from the short-term storage means, wherein the neural network learns the relationship between the learning image and the degree of urgency of the event represented by the image, and Means for recognizing the degree of urgency corresponding to the image captured by the means, the control means based on the recognition information output from the neural network, the image data transferred from the short-term storage means to the storage means A supervisory control device for controlling a frame rate of a video signal. A standard learning pattern storage unit for storing image data of an image photographed by the photographing unit, wherein the neural network uses the image data stored in the standard learning pattern storage unit as a teaching material; The supervisory control device according to claim 1, wherein the relationship between the captured image and the degree of urgency of the event represented by the image is re-learned. A neural network that outputs recognition information corresponding to an image photographed by the photographing means based on a learning result;
Control means for performing control based on the recognition information output from the neural network, the neural network learns the relationship between the learning image and the control content of the photographing means, and the photographing means A monitoring and control apparatus characterized in that the control means of the photographing means corresponding to a photographed image is recognized, and the control means controls the photographing means based on the recognition information. A neural network that outputs recognition information corresponding to an image photographed by the photographing means based on a learning result;
Control means for performing control based on the recognition information output from the neural network, wherein the neural network learns a relationship between a learning image and a display means for displaying the image, and the photographing means Monitoring means for recognizing a display means corresponding to an image photographed in step (a), wherein the control means selects a display means for displaying an image photographed by the photographing means based on the recognition information. . A neural network that outputs recognition information corresponding to an image photographed by the photographing means based on a learning result;
Control means for performing control based on the recognition information output from the neural network, the neural network learns the relationship between the learning image and the control content of the photographing means, and the photographing means A first neural network for recognizing the control contents of the photographing means corresponding to the photographed image, and a relationship between a learning image and a display means for displaying the image, and learning the image photographed by the photographing means. A second neural network for recognizing display means corresponding to the first neural network, wherein the control means controls the photographing means based on the recognition information of the first neural network, and the recognition information of the second neural network. A monitoring means for selecting a display means for displaying an image photographed by the photographing means on the basis of the image data. A neural network that outputs recognition information corresponding to an image photographed by the photographing means based on a learning result;
Control means for performing control based on the recognition information output from the neural network,
A short-term storage unit for temporarily storing image data of an image captured by the imaging unit,
Storage means for recording the image data transferred from the short-term storage means, wherein the neural network learns the relationship between the learning image and the control content of the imaging means, and is photographed by the imaging means Learning the relationship between the first neural network that recognizes the control content of the photographing means corresponding to the image and the learning image and the display means for displaying the image, and corresponding to the image photographed by the photographing means Learning the relationship between the second neural network recognizing the display means and the degree of urgency of the learning image and the event represented by the image, and determining the degree of urgency corresponding to the image photographed by the photographing means A third neural network for recognizing, the control means controlling the photographing means based on the recognition information of the first neural network, Selecting display means for displaying an image photographed by the photographing means based on the recognition information of the neural network, and transferring from the short-term storage means to the storage means based on the recognition information of the third neural network. And a frame rate of the image data to be controlled. A monitoring system comprising: a photographing unit for photographing a monitoring area; a display unit for displaying a video photographed by the photographing unit; and the monitoring control device according to claim 1. .

Images