JP6887352B2 - Monitoring system - Google Patents

Description

The present invention relates to a monitoring system for controlling the imaging apparatus for imaging an object existing in the monitored area.

Conventionally, there is a technique for automatically controlling the shooting direction of a shooting device. For example, Patent Document 1 describes a three-dimensional image in which a subject to be photographed is recognized from a wide-angle image captured by a sensor camera, the current position of the subject is sequentially measured, and a predetermined viewpoint of the camera for photographing is used as the origin. A technique for automatically driving and controlling a shooting camera according to the movement of a subject is disclosed based on information indicating the current position of the subject in space.

Japanese Unexamined Patent Publication No. 10-294890

In the technique described in Patent Document 1, if the sensor camera can recognize the subject, even if the current position of the subject is out of the shooting range when viewed from the direction of the current shooting camera, the subject is photographed so as to be photographed. The camera can be driven and controlled.
However, if the subject is outside the recognition range of the sensor camera, the sensor camera cannot recognize the subject, and as a result, the subject is within the range that can be photographed by changing the direction of the shooting camera. Even so, there is a problem that the subject cannot be photographed by the photographing camera.

The present invention has been made to solve the above-mentioned problems, and provides a monitoring system capable of controlling a photographing device so as to photograph a monitoring target that does not exist in the photographing range of the current photographing device. The purpose is.

The monitoring system according to the present invention is detected by a photographing device that captures a monitored area, a detection device that detects a signal that indicates a direction in which a monitored object is present, and a detection device that is transmitted from a specific position in the monitored area. any and monitored based on the signal indicating the direction in which there was, eh Bei a photographing direction control device for controlling the photographing direction of the photographing device, the sensing device, a signal for indicating a direction in which the monitoring target exists in space It is characterized by having a plurality of detection regions capable of detecting a signal indicating the direction in which the monitoring target exists even if it comes from the direction of .

According to the present invention, the photographing device can be controlled so as to photograph a monitoring target that does not exist in the photographing range of the current photographing device.

It is a figure which shows an example of the whole structure of the monitoring system which concerns on Embodiment 1. FIG. It is a flowchart for demonstrating an example of the operation which controls the photographing direction of the photographing apparatus in Embodiment 1. FIG. It is a figure for demonstrating an example of the whole structure of the detection apparatus of Embodiment 1. FIG. It is a figure which shows an example of the arrangement of the detection area in the detection device in Embodiment 1. FIG. It is a flowchart explaining an example of the operation of the detection apparatus in this Embodiment 1. It is a figure for demonstrating an example of the whole structure of the photographing apparatus equipped with the imaging direction control apparatus in Embodiment 1. FIG. It is a flowchart explaining an example of the operation which the imaging direction control apparatus of Embodiment 1 controls an imaging apparatus. FIG. 5 is a diagram showing an example of a conversion table used by the conversion unit when converting detection area information in the first embodiment. FIG. 5 is a schematic diagram showing an example of a shooting range and a shooting range of a surveillance camera in the first embodiment. It is a figure which shows an example of the installation environment of the surveillance camera in the example which applied the surveillance system of Embodiment 1 to the surveillance system which monitors a platform of a station and a railroad track. It is a figure for demonstrating an example of the operation which controls the shooting direction of a surveillance camera when a station employee finds a person to be monitored in the monitoring system as shown in FIG. 12A and 12B are diagrams showing an example of the hardware configuration of the photographing direction control device according to the first embodiment of the present invention. It is a figure which shows an example of the whole structure of the monitoring system which concerns on Embodiment 2. FIG. It is a flowchart explaining an example of the operation of the monitoring system of Embodiment 2. In the second embodiment, FIG. 15A is a diagram illustrating an example of a screen to be displayed on the display device when the reception unit of the display control device receives the proposed preset information, and FIG. 15A is an example of a confirmation instruction screen. 15B is an example of a presetting necessity instruction screen, and FIG. 15C is an example of a registration completion screen.

Hereinafter, in order to explain the present invention in more detail, a mode for carrying out the present invention will be described with reference to the accompanying drawings.
Embodiment 1.
FIG. 1 is a diagram showing an example of the overall configuration of the monitoring system according to the first embodiment.
In the monitoring system of the first embodiment, a site supervisor who exists in the monitored area and directly and visually monitors the monitored area, and an image obtained by shooting the monitored area from a place away from the monitored area, etc. By confirming, the monitoring work shall be performed by the remote monitor who monitors the monitored area.

The monitoring system includes a photographing device 101, a detection device 102, an instruction device 103, a display device 104, a recording device 105, and a control device 106. The photographing device 101 is equipped with a photographing direction control device 107.
The photographing device 101 photographs the monitored area as an image or a moving image. The photographing device 101 is, for example, a surveillance camera, a smartphone, a digital camera, or the like.
The photographing device 101 outputs the photographed image or video to the display device 104 and the recording device 105. The control device 106 controls the photographing device 101. For example, the orientation of the photographing device 101 is controlled. The control device 106 may be connected to the display device 104. Specific examples of the control device 106 include a controller, a remote controller, a mouse, and the like.
Further, the photographing device 101 changes the photographing direction based on the control of the photographing direction control device 107 to photograph the monitored area.

The shooting direction control device 107 controls the shooting direction of the shooting device 101 based on the information transmitted from the detection device 102. The details of the information transmitted from the detection device 102 and the specific operation of controlling the shooting direction of the shooting device 101 will be described later.
In the first embodiment, as shown in FIG. 1, the photographing device 101 is equipped with the photographing direction control device 107, but the present invention is not limited to this, and the photographing direction control device 107 is outside the photographing device 101. The photographing direction of the photographing device 101 may be controlled from the outside of the photographing device 101 via a network or the like.

The instruction device 103 is for transmitting a signal instructing the direction in which the monitoring target exists from a specific position in the monitoring target area. The instruction device 103 is, for example, a laser pointer or the like, and the site supervisor carries the instruction device 103. The specific position in the monitoring target area is an actual location in the monitoring target area, and is a place on a half line passing through the position of the monitoring target with the detection device 102 as one end, or a place in the vicinity of the half line. Is. The specific position is selected by the site supervisor according to the position where the monitoring target exists, and is not fixed.
Specifically, when the instruction device 103 is a laser pointer, for example, a site supervisor projects a light beam from the laser pointer onto a monitoring target existing in a monitoring target area, so that the light is reflected by the monitoring target. The emitted light beam is transmitted as a signal indicating the direction in which the monitored object exists. The specific position in this case is the position where the monitoring target exists.
Similarly, when the instruction device 103 is a laser pointer, for example, a site supervisor in the monitoring target area directly projects a light ray from the laser pointer onto the detection device 102 to monitor the light beam. Is transmitted as a signal indicating the direction in which is present. The specific position in this case is the position where the indicator 103 exists.

The detection device 102 detects the direction in which the monitoring target exists based on the signal transmitted by the instruction device 103. The detection device 102 is, for example, an optical sensor or the like.
Specifically, the detection device 102 has, for example, one or more detection regions each of which is composed of a sensor element.
For example, when the indicating device 103 is a laser pointer and the light beam from the laser pointer is projected onto the monitoring target, the detection device 102 detects the reflected light reflected from the monitoring target. By detecting in the area, the direction in which the monitoring target exists is detected. Further, for example, when the light beam from the laser pointer is directly projected onto the detection device 102, the detection device 102 detects the light beam projected from the laser pointer in the detection region, so that the laser pointer can be projected. The existing direction is detected as the direction in which the monitoring target exists.
The detection device 102 transmits information about the detection area that has detected the direction in which the monitoring target exists to the photographing direction control device 107 as detection area information.

The display device 104 displays an image or video output by the photographing device 101. The display device 104 is, for example, a display, a smartphone, a television, a video monitor, or the like. The display device 104 is assumed to be used in a place away from the monitored area, such as a monitoring room. The remote monitor monitors the monitored area by checking the image or video displayed on the display device 104.

The recording device 105 records an image or video output by the photographing device 101. The recording device 105 is, for example, a recorder, a smartphone, or the like.

The control device 106 controls the photographing device 101. Specifically, for example, the control device 106 controls operations such as zooming, focusing, panning, or tilting the photographing device 101. The control device 106 is, for example, a PC (Personal Computer) in which control software is installed, a controller, or the like.

The photographing device 101, the display device 104, the recording device 105, and the control device 106 are connected to each other so as to be able to communicate with each other.
Further, the photographing device 101 and the detecting device 102 are communicably connected to each other.
Further, the detection device 102 and the instruction device 103 are communicably connected to each other.

The operation of controlling the photographing direction of the photographing apparatus 101 in the monitoring system as shown in FIG. 1 will be described.
FIG. 2 is a flowchart for explaining an example of an operation of controlling the photographing direction of the photographing apparatus 101 in the first embodiment.
In FIG. 2, when the on-site supervisor finds a person to be monitored in the monitoring target area, the shooting direction of the shooting device 101 is set so that the range including the monitoring target is the shooting range of the shooting device 101. The operation of controlling the above will be described.
Further, in the following description, as an example, the detection device 102 is a spherical sensor, and the indicator device 103 is a laser pointer.

The instruction device 103 instructs the direction in which the person to be monitored exists (step ST201). Specifically, the on-site supervisor uses a portable instruction device 103 to project a light beam on a person to be monitored.

In step ST201, the detection device 102 detects the light beam projected from the instruction device 103 to the person to be monitored and reflected by the person as a signal indicating the direction in which the monitoring target exists. The detection device 102 transmits information about the detection area in which the signal is detected to the shooting direction control device 107 mounted on the shooting device 101 as detection area information (step ST202).

The shooting direction control device 107 controls the shooting direction of the shooting device 101 based on the information transmitted from the detection device 102 in step ST202 (step ST203). Then, the photographing device 101 photographs the monitored area in the photographing direction according to the control of the photographing direction control device 107.

By operating as described above, even if the monitoring target exists outside the current shooting range of the photographing device 101, the signal indicating the direction in which the monitoring target exists transmitted from the instruction device 103 is detected. Therefore, the shooting direction of the photographing device 101 can be controlled so that the monitoring target is included in the photographing range of the photographing device 101.

Hereinafter, details of the configuration and operation of the detection device 102 constituting the monitoring system and the photographing device 101 equipped with the photographing direction control device 107 will be described.

First, the detection device 102 will be described.
FIG. 3 is a diagram for explaining an example of the overall configuration of the detection device 102 of the first embodiment.
In FIG. 3A, when the detection device 102 directly detects the light beam projected from the instruction device 103 in the detection area 302, the direction in which the instruction device 103 exists is detected as the direction in which the monitoring target 305 exists. 3B shows the direction in which the monitoring target 305 exists by the detection device 102 detecting the light beam projected from the instruction device 103 to the monitoring target 305 and reflected by the monitoring target 305. Is shown.

In FIG. 3, the coordinate axis 301 is the xyz coordinate axis, and the origin is, for example, the center of the detection device 102. The z-axis direction is the vertical direction.
The detection device 102 has a plurality of detection areas 302. In FIG. 3, only one detection area 302 out of the plurality of detection areas 302 is shown. Further, in FIG. 3, as an example, each detection region is provided on the surface of the spherical detection device 102.
Each of the detection regions 302 is composed of, for example, a single sensor element for detecting a signal transmitted from the instruction device 103.
The detection device 102 may be provided with a plurality of detection regions so that the light beam projected in the monitoring target region can be detected regardless of the direction. The arrangement of the plurality of detection areas and the specific number of the plurality of detection areas will be described later.

The direction in which the monitoring target 305 exists is the detection target of the detection device 102.
As shown in FIG. 3A, the detection device 102 can detect the direction in which the monitoring target 305 exists by detecting the light beam projected directly from the instruction device 103 onto the detection device 102. Although the monitoring target 305 is not shown in FIG. 3A, the monitoring target 305 in this case exists on a half line passing through the position of the indicating device 103 with the detection device 102 as one end, or in the vicinity of the half line.
Further, as shown in FIG. 3B, when the detection device 102 detects the light beam projected from the instruction device 103 to the monitoring target 305 and reflected from the monitoring target 305, the detection device 102 presents the monitoring target 305. It becomes possible to detect the direction of the light.
The detection device 102 receives a light beam directly projected from the instruction device 103 to the detection device 102 or a light beam projected from the instruction device 103 to the monitoring target 305 as a signal instructing the direction in which the monitoring target 305 exists. The photographing device 101 is controlled to detect the light beam reflected from the target 305 and cause the photographing direction control device 107 to photograph the direction in which the monitoring target 305 exists.

Here, the photographing device 101 can photograph a wide field of view using, for example, a 120-degree wide-angle lens. Therefore, as long as the monitoring target 305 can be reflected in the field of view of the photographing device 101, the existing recognition function using, for example, the face detection of the subject, which is originally provided in the photographing device 101, is utilized thereafter. , It is possible to perform the desired monitoring work.
Therefore, the detection area 302 of the detection device 102 can be arranged so that, for example, the three directions in which 360 degrees in the horizontal direction can be divided into three substantially evenly as viewed from the center of the photographing device 101 can be distinguished. Further, for example, when the vertical direction is taken into consideration, the arrangement is such that 180 degrees in the vertical direction as viewed from the center of the photographing apparatus 101 can be divided into two directions substantially evenly. Specifically, when the detection area 302 is viewed from the center of the photographing apparatus 101, for example, 360 degrees in the horizontal direction is evenly divided into three, and 180 degrees in the vertical direction is evenly divided into two, as shown in FIG. In addition, it is a combination of two triangular pyramids. In this case, the number of detection areas 302 is six.
The arrangement of the detection areas 302 and the number of detection areas 302 of the detection device 102 are not limited to those in which two triangular pyramids are combined as described above, but after considering the field of view of the photographing device 101, that is, the wide angle. , The direction of the light beam indicating the direction in which the monitoring target 305 exists may be arranged as appropriate so as to be distinguishable. For example, any shape may be used as long as the detection region 302 is arranged so as to face the direction in which the photographing device 101 is desired, such as a shape in which four horizontal directions and three vertical directions can be distinguished and the detection regions 302 are twelve.

When the detection device 102 is composed of a spherical sensor as shown in FIG. 3, the center of the sphere, which is the intersection of each axis of the xyz coordinate axes 301, is used as a viewpoint, and the detection device 102 passes through the surface of the sphere. Assuming that the real space is viewed, the real space and the spherical surface of the detection device 102 can be associated and mapped.
In this way, the spherical surface of the detection device 102, which can be mapped in association with the real space, is divided so that a blind spot in the imaging range of the imaging device 101 does not occur and the resolution of the control coordinates is sufficient. It is composed of a plurality of sensor elements. For example, using the Universal Transverse Mercator projection, each region formed by dividing the surface of the sphere into 60 is designated as each detection region 302, and a single optical sensor element is assigned to each detection region 302 to configure the detection device 102. ..

The xyz coordinate axis 301 assists in drawing a space figure.
The detection area 302 shown in FIG. 3A shows, among the plurality of detection areas 302 included in the detection device 102, the detection area 302 in which the light beam projected directly from the instruction device 103 to the detection device 102 is detected. There is.
Of the plurality of detection areas 302 included in the detection device 102, the detection area 302 shown in FIG. 3B detects a light beam projected from the instruction device 103 to the monitoring target 305 and reflected from the monitoring target 305. The detection area is shown.
The detection device 102 uses information about the detection area 302 that has detected a light beam indicating the direction in which the monitoring target 305 exists, such as the detection area 302 shown in FIG. 3A or FIG. 3B, as detection area information in the photographing direction. It is transmitted to the control device 107.

The shooting direction control device 107 receives the detection area information transmitted from the detection device 102 and determines the shooting direction of the shooting device 101. For example, when the lens of the photographing device 101 is a wide-angle lens of 120 degrees, the photographing direction control device 107 can divide the field of view of the photographing device 101 of 360 degrees into three, so that the accurate position information of the monitoring target 305 is not specified. However, it is possible to determine the shooting direction in which the subject can be captured within the field of view of the shooting device 101.

Since the light rays detected by the detection device 102 may include ambient light and the like, the detection area information transmitted from the detection device 102 may include erroneous detection area information due to the detection of the disturbance light and the like. .. Therefore, for example, by setting the light rays that indicate the direction in which the monitoring target 305 exists as a specific pattern (blinking), the detection device 102 distinguishes between the light rays that indicate the direction in which the monitoring target 305 exists and the ambient light. The accuracy is significantly improved, and the detection accuracy is improved.
The detection device 102 notifies the photographing direction control device 107 of the detection area information when it receives a light ray having a predetermined pattern, and discards the information when it receives other light rays. In this way, the detection device 102 can determine the validity of the received light beam and reduce erroneous detection. The detailed configuration and operation of the photographing direction control device 107 will be described later.

The mapping between the real space and the surface of the detection device 102 described above can be performed by determining a predetermined viewpoint even when the detection device 102 is composed of a sensor other than a spherical sensor.

FIG. 5 is a flowchart illustrating an example of the operation of the detection device 102 according to the first embodiment.
That is, FIG. 5 is a flowchart illustrating an example of the detailed operation of step ST202 of FIG.
The detection device 102 detects a light beam emitted from the instruction device 103 as a signal indicating the direction in which the monitoring target 305 exists in the detection area 302 (step ST501).

The detection device 102 transmits the information regarding the detection area 302 in which the light beam is detected in step ST501 to the photographing direction control device 107 as the detection area information (step ST502).

Next, a photographing device 101 equipped with the photographing direction control device 107 will be described.
FIG. 6 is a diagram for explaining an example of the overall configuration of the photographing device 101 equipped with the photographing direction control device 107 in the first embodiment.
As shown in FIG. 6, the photographing device 101 is equipped with a photographing direction control device 107. Further, as shown in FIG. 6, the photographing apparatus 101 includes a photographing unit 1011, a focus matching processing unit 1012, and a turning processing unit 1013.

The photographing unit 1011 photographs an image or a moving image of a monitored area. The photographing unit 1011 transmits the photographed image or video of the monitored area to the display device 104, the recording device 105, and the control device 106 (see FIG. 1).
The focus matching processing unit 1012 focuses the photographing unit 1011 based on the control from the photographing direction control device 107.
The swivel processing unit 1013 swivels the photographing unit 1011 based on the control from the photographing direction control device 107.

The shooting direction control device 107 is communicably connected to the detection device 102, and receives detection area information from the detection device 102 by a receiving unit (not shown) included in the shooting direction control device 107.
The shooting direction control device 107 controls the direction of the shooting device 101 in the direction in which the monitoring target 305 exists when the detection device 102 is used as a reference, based on the detection area information received from the detection device 102 by the receiving unit.
The shooting direction control device 107 includes a shooting control unit 1071 and a conversion unit 1072.
The imaging control unit 1071 controls the operations of the focus matching processing unit 1012 and the turning processing unit 1013 based on the parameters acquired from the control device 106 or the conversion unit 1072. The control device 106 converts, for example, information indicating up, down, left, right, etc. so as to match the I / F (interface) required by the photographing control unit 1071, and photographs the converted parameters as control information. Output to control unit 1071. The control based on the information from the control device 106 assumes, for example, manual control performed by a remote monitor operating the controller.
The conversion unit 1072 acquires the detection area information from the detection device 102 from the reception unit, converts the detection area information so as to match the I / F required by the imaging control unit 1071, and converts the parameters. Is output to the photographing control unit 1071 as control information.
The photographing control unit 1071 controls the operation of the focus matching processing unit 1012 or the turning processing unit 1013 based on the parameters acquired from the control device 106 or the conversion unit 1072, so that the photographing unit 1011 can be rotated. The operation is controlled.

FIG. 7 is a flowchart illustrating an example of an operation in which the photographing direction control device 107 of the first embodiment controls the photographing device 101.
The receiving unit of the shooting direction control device 107 receives the detection area information transmitted from the detection device 102 (step ST701). The receiving unit outputs the received detection area information to the conversion unit 1072.

The conversion unit 1072 converts the detection area information output from the reception unit in step ST701 so as to match the I / F required by the imaging control unit 1071, and outputs the converted parameters as control information to the imaging control unit ( Step ST702). Specifically, the conversion unit 1072 converts the detection area information based on the conversion table as shown in FIG. For example, when the conversion unit 1072 acquires the detection area information indicating the detection area A received from the detection device 102 via the reception unit, the conversion unit 1072 uses the detection area information as a reference when the pan angle is 0 degrees from the reference and the tilt angle is the reference. It is converted into control information of 45 degrees and output to the photographing control unit 1071. The detection area A or the like may be specified by coordinates by providing coordinate axes in the detection device 102 or the real space. The conversion table as shown in FIG. 8 is created in advance and stored in a place where the photographing direction control device 107 can be referred.

The photographing control unit 1071 controls the operation of the focus matching processing unit 1012 or the turning processing unit 1013 based on the control information output from the conversion unit 1072 in step ST702. As a result, the operation of the photographing unit 1011 such as the turning process is controlled (step ST703).
The photographing unit 1011 transmits the captured image or video to the display device 104 and the recording device 105.

Hereinafter, application examples of the monitoring system according to the first embodiment of the present invention will be described with reference to FIGS. 9 to 11.
In the following, for example, an example in which the monitoring system of the first embodiment of the present invention is applied to a monitoring system for monitoring a platform of a station and a railroad track will be shown. In this case, the platform of the station and the railroad track are the monitoring target areas. In addition, the on-site supervisor is a station employee 1001 at the platform of the station, and the remote observer is in the management room or the like and confirms the image or video displayed on the display device installed in the management room or the like. , A station employee who monitors the platform of the station.
In the monitoring system, the surveillance camera 801 which is the photographing device 101 is installed so as to be able to turn the platform 902 and the track 901 of the station at a position where the image can be taken, and the station staff 1001 who is the site supervisor is the laser pointer 1002 which is the indicating device 103. Shall carry.
Further, it is assumed that a spherical sensor, which is a detection device 102, is installed in the vicinity of the surveillance camera 801. In FIGS. 9 to 11 used in the following description, the description of the detection device 102 is omitted.

FIG. 9 is a schematic view showing an example of the shooting range 802 and the shooting range 803 of the surveillance camera 801 in the first embodiment. The upper figure in FIG. 9 is a diagram showing a photographing range 802, and the lower figure in FIG. 9 is a diagram showing a photographing range 803.
Here, the shooting range 802 means the shooting range of the surveillance camera 801 at the current installation position. Further, the shootable range 803 refers to a shooting range that is outside the shooting range of the surveillance camera 801 at the current installation position, but can be shot by performing control such as turning the surveillance camera 801.
The surveillance camera 801 is a photographing device 101 capable of camera control of zoom, focus, pan, and tilt.

FIG. 10 is a diagram showing an example of the installation environment of the surveillance camera 801 in the example in which the surveillance system of the first embodiment is applied to the surveillance system for monitoring the platform 902 of the station and the railroad track 901.
As shown in FIG. 10, the surveillance camera 801 is installed at a position where the status of the train track 901 and the platform 902 of the station can be monitored.

FIG. 11 is a diagram for explaining an example of an operation in which the shooting direction of the surveillance camera 801 is controlled when the station employee 1001 discovers the monitored person 1003 in the monitoring system as shown in FIG.
The surveillance camera 801 photographs a certain direction (shown by a in FIG. 11) at time t1 and a different direction (shown by b in FIG. 11) from time t1 at time t2.
It is assumed that at time t1, the surveillance camera 801 is facing the direction of a, and the station employee 1001 at the platform 902 finds a suspicious person to be monitored. The suspicious person is the person to be monitored 1003.
The station employee 1001 projects light on the monitored person 1003 with a portable laser pointer 1002 (the projected light beam is shown by c in FIG. 11).

Then, the surveillance camera 801 that has been photographing the direction a in FIG. 11 turns from the photographing range that photographs the direction a in FIG. 11 to the position that becomes the photographing range that photographs the direction b in FIG.
Specifically, the detection area included in the detection device 102 detects the light beam from the laser pointer 1002 reflected by the monitored person 1003, and the detection device 102 transmits the detection area information to the photographing direction control device 107.
The shooting direction control device 107 controls the focus matching processing unit 1012 or the turning processing unit 1013 of the surveillance camera 801 so that the direction in which the monitored person 1003 exists is the shooting direction based on the detection area information. As a result, the shooting direction of the surveillance camera 801 is controlled, and the surveillance target person 1003 is included in the shooting range of the surveillance camera 801.

The light beam emitted by the laser pointer 1002 may be visible light or invisible light, and may be constantly lit or blinking. That is, the apparent color of the light beam does not matter, and the light projection pattern may be arbitrary. Further, the laser pointer 1002 may emit one or more light rays at the same time, or may emit light at non-simultaneous times.

As described above, in the monitoring system of the first embodiment, for example, a user of the monitoring system, such as a site supervisor, uses the instruction device 103 to directly project light on the detection device 102 or cast the light on the monitoring target. By a simple means of illuminating, the photographing device 101 can be directed in the direction in which the monitored object exists. As a result, for example, when the on-site supervisor grasps the direction in which the monitoring target exists, the shooting range of the photographing device 101 can be quickly directed to the monitoring target, and the imaging device 101 can be moved to a remote location. It is possible to quickly transmit a reflected image or video to be monitored to a certain display device.

In the above-mentioned conventional technique, when a monitoring target exists outside the shooting range of the sensor camera that shoots a wide-angle image, in order to shoot the monitoring target with the shooting camera, for example, a site supervisor near the sensor camera Recognizes the monitoring target and notifies the remote monitor who is located away from the monitoring target area that the monitoring target has been recognized. However, in this case, when the on-site supervisor conveys information on the monitoring target, there is a possibility that a discrepancy in recognition or a human error may occur.
In addition, the on-site supervisor cannot confirm whether or not the monitoring target is included in the shooting range of the sensor camera unless the display device is installed near the installation location of the sensor camera.
As described above, in the prior art, when a monitoring target that is within the shooting range of the shooting device but is not included in the shooting range of the current shooting device is recognized, the range including the monitoring target is shot. In some cases, it may not be possible to control the imaging device so that it is within the imaging range of the device. In that case, it takes a lot of effort to control the imaging device, and the imaging device is set so that the imaging range of the imaging device includes the monitoring target. It becomes difficult to adjust the orientation accurately.
Further, in the prior art, even if the monitoring target exists within the shooting range of the sensor camera, if the monitoring target does not meet the predetermined conditions, the monitoring target cannot be recognized and is used for shooting. The camera cannot be aimed at the surveillance target.

On the other hand, in the monitoring system of the first embodiment of the present application, as described above, the instruction device 103 carried by the site supervisor is used to directly project light on the detection device 102, or to project light on the monitored object. By doing so, it is possible to indicate the direction in which the monitoring target exists and control the photographing direction of the photographing device 101.
At that time, since the on-site supervisor does not need to explain the direction in which the monitoring target exists to the remote observer, there is a discrepancy in recognition between the on-site supervisor and the remote observer, or The possibility of human error can be suppressed.
In addition, since the on-site supervisor uses the instruction device 103 to instruct the direction in which the monitoring target exists, the monitoring target is determined and instructed by human judgment, and the monitoring target is determined more flexibly or more accurately. In addition, the monitoring target to be monitored can be captured by the photographing device 101.

12A and 12B are diagrams showing an example of the hardware configuration of the photographing direction control device 107 according to the first embodiment of the present invention.
In the first embodiment of the present invention, the imaging control unit 1071 is realized by the processing circuit 111. That is, the photographing direction control device 107 includes a processing circuit 111 for controlling the photographing direction of the photographing device 101 based on the detection area information received from the detection device 102.
The processing circuit 111 may be dedicated hardware as shown in FIG. 12A, or may be a CPU (Central Processing Unit) 116 that executes a program stored in the memory 115 as shown in FIG. 12B.

When the processing circuit 111 is dedicated hardware, the processing circuit 111 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable). Gate Array) or a combination of these is applicable.

When the processing circuit 111 is the CPU 116, the function of the photographing control unit 1071 is realized by software, firmware, or a combination of software and firmware. That is, the photographing control unit 1071 is realized by a processing circuit such as an HDD (Hard Disk Drive) 112, a CPU 116 that executes a program stored in a memory 115, and a system LSI (Large-Scale Integration). Further, the program may be stored in the processing circuit 111. Further, it can be said that the program stored in the HDD 112, the memory 115, or the like causes the computer to execute the procedure or method of the shooting control unit 1071. Here, the memory 115 is, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Online Memory), an EEPROM (Electrically Memory), etc. , Non-volatile or volatile semiconductor memory, magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versaille Disc) and the like.

Further, the photographing direction control device 107 includes an input interface device 113 and an output interface device 114 that communicate with a device such as the photographing device 101 or the detection device 102. The receiving unit is composed of, for example, the input interface device 113.

As described above, according to the first embodiment, the monitoring system is a signal indicating the direction in which the photographing device 101 that captures the monitored area and the monitored object transmitted from a specific position in the monitored area exist. Since the detection device 102 for detecting the above and the shooting direction control device 107 for controlling the shooting direction of the shooting device 101 based on the signal indicating the direction in which the monitoring target detected by the detection device 102 exists are provided. The imaging device can be controlled to capture a monitored object that does not exist in the imaging range of the current imaging device.

Embodiment 2.
In the first embodiment, the detection device 102 detects the instruction from the instruction device 103, and the imaging direction control device 107 controls the imaging direction of the imaging device 101 based on the detection area information detected by the detection device 102. I was doing it.
In the second embodiment, an embodiment in which a frequent shooting direction is learned from the history of the shooting direction of the shooting device 101 controlled by the shooting direction control device 107 will be described.

FIG. 13 is a diagram showing an example of the overall configuration of the monitoring system according to the second embodiment.
In FIG. 13, the same reference numerals are given to the same configurations as those described with reference to FIG. 1 in the first embodiment, and duplicate description will be omitted. In the second embodiment as well, as an example, as shown in FIG. 13, the photographing device 101a is equipped with the photographing direction control device 107, but the imaging direction control device 107 is not limited to this. It may be provided outside the 101a.
Since the configuration of the photographing direction control device 107 is the same as the configuration described with reference to FIG. 6 in the first embodiment, the same reference numerals are given and duplicate description will be omitted.

The monitoring system shown in FIG. 13 differs from the monitoring system shown in FIG. 1 in the detailed configurations of the photographing device 101a, the display device 104a, and the recording device 105a. The photographing device 101 in the monitoring system of the first embodiment did not have a metadata (for example, pan angle, tilt angle, zoom value, etc.) distribution function, whereas the photographing device 101a in the monitoring system of the second embodiment did not have a metadata distribution function. Is different in that it has a metadata distribution function. Further, the display device 104 in the monitoring system of the first embodiment has a preset function of controlling the shooting device at the preset position by setting the shooting direction (pan angle, tilt angle, zoom value, etc.) of the shooting device specified in advance as a preset position. The display device 104a in the monitoring system of the second embodiment is different in that it is provided with the display control device 108 having a preset function, whereas the display control device 108 is not provided. Further, the recording device 105 in the monitoring system of the first embodiment does not have a learning function for learning the shooting direction with high frequency, whereas the recording device 105a in the monitoring system of the second embodiment has a learning function. The point to prepare is different.

The photographing device 101a has a metadata distribution function. The metadata may be distributed as a stream in the same manner as the video data, or the metadata may be embedded in the video stream. Specifically, H.A., which distributes a metastream, superimposes a metadata value as image data on video data, and stores it in APPn, which is a header area of JPEG. 264 | MPEG-4 Stored in SEI, which is the header area of AVC, etc.

The display device 104a is equipped with a display control device 108.
In the second embodiment, as an example, as shown in FIG. 13, the display device 104a is equipped with the display control device 108, but the present invention is not limited to this, and the display control device 108 is outside the display device 104a. It may be provided.
The display control device 108 exposes an external I / F (interface) so that preset candidates can be proposed from an external device (for example, a recording device 105a). Further, the preset position may be saved by the display control device 108 or may be recorded by the recording device 105a.
As shown in FIG. 13, the display control device 108 includes a control unit 1081 and a reception unit 1082.
The control unit 1081 executes various instructions from the reception unit 1082.
The reception unit 1082 receives various instructions from the external I / F (recording device 105a, remote monitor, etc.) and transmits them to the control unit 1081 as needed.
The reception unit 1082 receives information on preset candidates (hereinafter, referred to as proposed preset information) transmitted from the recording device 105a, and causes the display device 104a to display a screen or the like for proposing preset candidates. Further, when the reception unit 1082 receives the instruction information (first instruction information) for confirming the contents of the preset candidate by the remote monitor selecting the confirmation instruction screen, the reception unit 1081 sends the control unit 1081 the shooting direction. The imaging control unit 1071 of the control device 107 is transmitted from the recording device 105a (details will be described later) with the proposed preset information that has been received, and the display device 104a is made to display a screen or the like for selecting whether or not presetting is performed. Further, the reception unit 1082 transmits the instruction information (second instruction information) to the effect that the presetting or not the presetting received from the display device 104a is performed to the recording device 105a.

The recording device 105a includes a proposed preset transmission / reception control unit 1051, a recording control unit 1052, a recording unit 1053, and a proposed preset calculation unit 1054.
The proposed preset transmission / reception control unit 1051 transmits the proposed preset information output from the proposed preset calculation unit 1054 to the display control device 108. Further, the proposed preset transmission / reception control unit 1051 receives the second instruction information transmitted from the display control device 108 and outputs the second instruction information to the proposed preset calculation unit 1054.

The recording control unit 1052 receives the control information of the shooting direction of the shooting device 101a transmitted from the shooting direction control device 107, and records it in the recording unit 1053 as learning data.
The recording unit 1053 records the control information of the photographing direction of the photographing apparatus 101a as learning data based on the control of the recording control unit 1052.
In the second embodiment, when the shooting control unit 1071 of the shooting direction control device 107 controls to change the shooting direction of the shooting device 101a, the image data or video data of the shooting direction after the change, and the image data thereof. The control parameters are transmitted to the recording device 105a. For example, the image data or the video data and the control parameters are each delivered as a stream from the photographing device 101a.

The proposed preset calculation unit 1054 learns the shooting direction of the shooting device 101a based on the learning data recorded in the recording unit 1053, and determines the frequently shooting direction as the shooting direction proposed as a preset candidate. The learning data recorded in the recording unit 1053 is data including control information of the shooting direction as described above, and specifically, is data including pan angle, tilt angle, zoom value, etc., which are parameters at the time of control. is there.
The proposed preset calculation unit 1054 outputs the control information of the shooting direction proposed as a preset candidate to the proposed preset transmission / reception control unit 1051 as the proposed preset information.

The operation of the monitoring system of the second embodiment will be described.
FIG. 14 is a flowchart illustrating an example of the operation of the monitoring system according to the second embodiment.
In the second embodiment, as described above, when the photographing control unit 1071 of the photographing direction control device 107 controls the photographing direction with respect to the photographing device 101a, the control information of the photographing direction is transmitted to the recording control unit of the recording device 105a. Send to 1052.
The operation described below with reference to FIG. 14 shall be performed, for example, at the timing of receiving an instruction from the on-site supervisor. Note that this is only an example, and the operations described below may be performed at other timings.

The recording control unit 1052 of the recording device 105a receives the control information of the shooting direction transmitted from the shooting direction control device 107 (step ST1401), and causes the recording unit 1053 to record it as learning data (step ST1402).

The proposed preset calculation unit 1054 learns the frequency of the shooting direction based on the learning data recorded in the recording unit 1053 (step ST1403).
The proposed preset calculation unit 1054 learns the frequency of the photographing direction of the photographing apparatus 101a by machine learning such as clustering.
Specifically, the proposed preset calculation unit 1054 learns the frequency of the shooting direction based on the learning data recorded in the recording unit 1053. The frequency of the shooting direction is, for example, the specified number of times in the last week when the corresponding shooting direction is specified, the total specified number of times when the corresponding shooting direction is specified, the continuous specified time when the corresponding shooting direction is specified, and the like. .. The proposed preset calculation unit 1054 classifies the shooting direction that appears relatively frequently as a result of learning as the “high frequency” shooting direction.

As a result of learning in step ST1403, the proposed preset calculation unit 1054 determines whether or not there is a shooting direction classified as high frequency (step ST1404).

If it is determined in step ST1404 that there is no shooting direction classified as high frequency (in the case of “NO” in step ST1404), the process returns to step ST1401.

When it is determined in step ST1404 that there is a shooting direction classified as high frequency (when “YES” in step ST1404), the proposed preset calculation unit 1054 determines the control information of the shooting direction determined to be classified as high frequency. Is output to the proposed preset transmission / reception control unit 1051 as the proposed preset information. For example, when the proposed preset calculation unit 1054 has a plurality of shooting directions determined to be classified as high frequency, the control information of the shooting direction determined to be the most frequent is used as the proposed preset information.
When the proposed preset transmission / reception control unit 1051 acquires the proposed preset information from the proposed preset calculation unit 1054, the proposed preset transmission / reception control unit 1051 transmits the proposed preset information to the display control device 108 (step ST1405).

When the proposed preset information is received from the recording device 105a, the reception unit 1082 of the display control device 108 confirms with the display device 104a whether or not to confirm the contents of the frequently taken shooting directions based on the received proposed preset information. Display the confirmation instruction screen for accepting the designation of from the remote monitor. The confirmation instruction screen is displayed using, for example, a GUI (Graphical User Interface).
Here, FIG. 15 is a diagram illustrating an example of a screen to be displayed on the display device 104a when the reception unit 1082 of the display control device 108 receives the proposed preset information in the second embodiment, and FIG. 15A is a diagram. , This is an example of a confirmation instruction screen. The screen examples shown in FIGS. 15B and 15C will be described later.

When the confirmation instruction screen as shown in FIG. 15A is displayed on the display device 104a, the remote monitor confirms the confirmation instruction screen and determines whether or not to frequently confirm the contents in the shooting direction.
Then, the remote monitor inputs an instruction to confirm the content or not to confirm the content from the confirmation instruction screen using an input device (not shown) such as a touch panel, a mouse, or a keyboard.
For example, in the confirmation instruction screen of FIG. 15A, the remote observer inputs by selecting "Yes" when confirming the content and "No" when not confirming the content on the touch panel.
The reception unit 1082 of the display control device 108 receives the first instruction information input from the input device to confirm the contents or not to confirm the contents, and receives the proposed preset information received from the recording device 105a. , Is transmitted to the control unit 1081.

The reception unit 1082 determines whether or not the first instruction information for confirming the contents has been received (step ST1406).
In step ST1406, when the first instruction information for confirming the content is not received (when "NO" in step ST1406), that is, when the first instruction information for not confirming the content is accepted. Return to step ST1401. When the remote monitor receives the first instruction information that the content is not confirmed, the reception unit 1082 discards the proposal preset information received from the proposal preset transmission / reception control unit 1051 of the recording device 105a in step ST1405. ..

When the first instruction information for confirming the content is received in step ST1406 (when “YES” in step ST1406), the reception unit 1082 tells the control unit 1081 from the recording device 105a in step ST1405. The received proposed preset information is transmitted, and the control unit 1081 transmits the same information to the shooting control unit 1071 of the shooting direction control device 107.
When the shooting control unit 1071 receives the proposed preset information from the control unit 1081, the shooting control unit 1071 controls the shooting direction of the shooting device 101a so that the shooting direction is based on the received proposed preset information. Since the specific operation is the same as the shooting direction control operation described in the first embodiment, duplicate description will be omitted (see step ST703 in FIG. 7).
When the shooting direction of the shooting device 101a is controlled by the shooting control unit 1071, the shooting unit 1011 of the shooting device 101a shoots an image or video of the monitored area in the shooting direction based on the control of the shooting direction control device 107. The image or video and control parameters are transmitted to the display device 104a and the recording device 105a.
In the second embodiment, at this time, in order to prevent the control parameter at the time of the control instruction according to the confirmation content from becoming learning data, the photographing unit 1011 of the photographing apparatus 101a records, for example, the fact together with the image or the video. It transmits to the device 105a. It should be noted that this is only an example, and it suffices if it can be determined that it is a control parameter at the time of control by the proposed preset.

The display device 104a displays an image or video transmitted from the photographing unit 1011.
The reception unit 1082 of the display control device 108 causes the display device 104a to display a presetting necessity instruction screen for confirming whether or not to preset the shooting direction in which the displayed image or video is shot.

FIG. 15B is a diagram showing an example of a presetting necessity instruction screen to be displayed on the display device 104a by the reception unit 1082 of the display control device 108 in the second embodiment. The reception unit 1082 causes the display device 104a to display the presetting necessity instruction screen as shown in FIG. 15B on, for example, a pop-up screen.
The remote monitor needs to check the image or video currently displayed on the display device 104a and preset the shooting direction in which the image or video was shot, or give an instruction not to preset. Input using the input device from the rejection instruction screen.
For example, in the presetting necessity instruction screen of FIG. 15B, the remote monitor inputs by selecting "Yes" when performing presetting and "No" when not performing presetting on the touch panel.
The reception unit 1082 of the display control device 108 receives the second instruction information input from the input device to the effect that the presetting is performed or the presetting is not performed.
The reception unit 1082 transmits to the recording device 105a the second instruction information input from the input device to the effect that the presetting is performed or the presetting is not performed. At this time, the reception unit 1082 adds the second instruction information to the proposal preset information received from the proposal preset transmission / reception control unit 1051 of the recording device 105a in step ST1405, and transmits the second instruction information to the recording device 105a.
The reception unit 1082 transmits the second instruction information input from the input device to the effect that the presetting is performed or the presetting is not performed to the recording device 105a, and then displays the registration completion screen on the display device 104a. Let me.
FIG. 15C is a diagram showing an example of a registration completion screen displayed on the display device 104a by the reception unit 1082 in the second embodiment. As a default setting, the preset name may be set by the reception unit 1082 when the second instruction information for presetting is received, or the remote observer presets the remote observer. When inputting the second instruction information to the effect that the above is performed, it may be input each time.

The proposed preset transmission / reception control unit 1051 of the recording device 105a determines whether or not the second instruction information for presetting is received from the display control device 108 (step ST1407).
In step ST1407, when the second instruction information indicating that presetting is not performed is not received (when “NO” in step ST1407), that is, when the second instruction information indicating that presetting is not performed is received. , The proposed preset transmission / reception control unit 1051 feeds back to the proposed preset calculation unit 1054 that the presetting is not performed (step ST1411), and proceeds to step ST1409.

When the second instruction information for presetting is received in step ST1407 (when “YES” in step ST1407), the proposed preset transmission / reception control unit 1051 performs presetting to the proposed preset calculation unit 1054. Feedback to that effect (step ST1408).

When feedback is given to the effect that presetting is to be performed or not to be preset, the proposed preset calculation unit 1054 uses the proposed preset information as the proposed preset information in step ST1404, for example, in the case of feedback to perform presetting. The control information of the device 101a is excluded from the learning target. This is only an example, and the correct answer information selected by the remote monitor may be fed back to the learning system to improve efficiency.

In the above description, when the proposed preset calculation unit 1054 has a plurality of shooting directions determined to be classified as high frequency, the proposed preset calculation unit 1054 uses the control information of the shooting direction of the shooting device 101a determined to be the most frequent. The proposed preset information is not limited to this, and for example, the proposed preset calculation unit 1054 may create the proposed preset information for all shooting directions determined to be classified at high frequency. In this case, in the display control device 108, for example, on the confirmation instruction screen, it is possible to accept the designation of which shooting direction the proposed preset information is to be confirmed, and the remote observer can arbitrarily specify and control a single unit. Alternatively, the desired proposed preset information can be arbitrarily selected from a plurality of images or videos.

As described above, according to the second embodiment, the monitoring system includes a display control device 108 or a recording device 105a having a preset function capable of designating and holding control information of the shooting direction in advance, and controls the shooting direction. A recording control unit 1052 that receives control information of the imaging direction of the imaging device 101a controlled by the device 107, and a recording unit that records the control information of the imaging direction of the imaging device 101a received by the recording control unit 1052 as learning data. A recording device 105a having a 1053 and a proposed preset calculation unit 1054 that learns the shooting direction of the shooting device 101a based on the learning data recorded by the recording unit 1053 and sets a high frequency shooting direction as a preset candidate is provided. Since it is configured, it is possible to automatically extract and preset the shooting directions that are frequently specified based on the past control status.
Thereby, for example, the shooting range overlooked by the remote monitor can be specified from the shooting direction designation information of the on-site supervisor. In the monitoring system, the importance of the shooting direction is learned from the specified information of the shooting direction of the on-site supervisor, and the shooting direction in the overlooked range that is not preset by the remote monitor is also proposed to be preset to be monitored. It is possible to reduce the situation of missing.

In the present invention, within the scope of the invention, it is possible to freely combine each embodiment, modify any component of each embodiment, or omit any component in each embodiment. ..

101, 101a imaging device, 102 detection device, 103 instruction device, 104, 104a display device, 105, 105a recording device, 106 control device, 107 imaging direction control device, 108 display control device, 111 processing circuit, 112 HDD, 113 inputs Interface device, 114 output interface device, 115 memory, 116 CPU, 302 detection area, 305 monitoring target, 801, 801a surveillance camera, 802 shooting range, 803 shooting range, 901 line, 902 home, 1001 site supervisor, 1002 laser Pointer, 1003 Monitored person, 1011 Imaging unit, 1012 Focus matching processing unit, 1013 Turning processing unit, 1051 Transmission / reception control unit, 1052 Recording control unit, 1053 Recording unit, 1054 Preset calculation unit, 1071 Imaging control unit, 1072 Conversion unit, 1081 control unit, 1082 reception unit.

Claims (9)

An imaging device that captures the monitored area and
A detection device that detects a signal indicating the direction in which the monitoring target exists, which is transmitted from a specific position in the monitoring target area, and
Said sensing device based on the signal indicating the direction in which there is the monitoring target is detected, e Bei a photographing direction control device for controlling the photographing direction of the photographing device,
The detection device has a plurality of detection regions capable of detecting a signal indicating the direction in which the monitoring target exists regardless of the direction in which the signal indicating the direction in which the monitoring target exists comes from any direction in the space.
A monitoring system characterized by that. Further comprising an instruction unit for transmitting a signal indicating the direction in which the monitoring target is present from a specific position of the monitored area,
The sensing device monitoring system of claim 1, wherein the detecting a signal indicating the direction in which the monitoring target originated by the pointing device is present. The instruction device, the a specific position the position where the monitoring target is present in the monitored area, according to claim 2, wherein the transmitting a signal indicating the direction in which the monitoring target is present from the specific position Monitoring system. The pointing device is for emitting a light beam, the light beam is projected on the monitored, the light rays reflected by the monitoring target, that transmits a signal indicating the direction in which the monitoring target is present The monitoring system according to claim 3, wherein the monitoring system is characterized. The monitoring system according to claim 2, wherein the instruction device emits a light ray, and emits a light ray directly projected onto the detection device as a signal instructing a direction in which a monitoring target exists. A recording control unit that receives control information of the shooting direction of the shooting device, which is controlled by the shooting direction control device, and a recording control unit.
A recording unit that records control information of the imaging direction of the imaging device received by the recording control unit as learning data, and a recording unit.
Claims 1 to 1 include a recording device including a proposed preset calculation unit that learns the shooting direction of the shooting device based on the learning data recorded by the recording unit and uses a high-frequency shooting direction as a preset candidate. The monitoring system according to any one of 5. Further equipped with a display control device that receives an instruction as to whether or not to preset the shooting direction specified in advance based on the preset candidate.
When the display control device receives a second instruction information for presetting a predetermined shooting direction based on the preset candidate, the display control device or the recording device is based on the preset candidate in advance. The monitoring system according to claim 6, wherein the control information of the designated shooting direction is held as the information of the preset position. The proposed preset calculation unit is characterized in that it learns the shooting direction of the shooting device based on the second instruction information for instructing the shooting direction of the shooting device as the preset candidate to be the preset position and the preset position. The monitoring system according to claim 6. When the information on the preset candidate determined by the proposed preset calculation unit is received and the first instruction information for confirming the content of the preset candidate is received, the shooting direction control device is referred to the preset candidate. Further equipped with a display control device having a reception unit for transmitting information,
When the shooting direction control device receives information about the preset candidate from the reception unit, the shooting direction control device controls the shooting direction of the shooting device so that the shooting direction is based on the preset candidate.
When the reception unit receives the second instruction information for presetting the shooting direction controlled by the shooting direction control device, the receiving unit transmits the second instruction information to the proposed preset calculation unit.
The monitoring system according to claim 6, wherein the proposed preset calculation unit presets the shooting direction of the shooting device as the preset candidate.

Images