JP7356524B2 - monitoring display device - Google Patents

Description

The present invention relates to a monitoring display device for monitoring a monitoring space.

In recent years, the importance of security by security guards has increased in order to prevent crimes at facilities such as airports, stations, commercial facilities (department stores, supermarkets, etc.), entertainment halls, and corporate offices. The labor shortage is becoming more and more serious. In order to solve the shortage of security guards, a monitoring display device has been developed that monitors the monitoring space of each facility while displaying an image of a character imitating a security guard.

For example, Patent Document 1 discloses a device equipped with a video output device that is equipped with a screen that matches the shape of a person's upper body and projects an image of a security guard in accordance with the shape of the screen. This device equipped with a video output device projects an image of a security guard on a screen when a person is detected by a human sensor that detects whether there is a person within a predetermined range.

Japanese Patent Application Publication No. 2011-150221

In a surveillance display device that monitors a surveillance space while displaying an image of a character imitating a security guard, in order to give people in the surveillance space a sense of being watched and a sense of security that they are being watched, There is a need to display an image that looks more like a real security guard.

An object of the present invention is to provide a surveillance display device that can display an image that looks more like a real security guard when monitoring a surveillance space while displaying an image of a character imitating a security guard.

In order to solve such problems, the present invention includes a display section that displays an image of a character imitating a security guard standing guard, a detection section that detects a person in a surveillance space, and a display section that displays an image of a character imitating a security guard on guard duty, a detection section that detects a person in a surveillance space, and a display unit that displays a character image imitating a security guard on guard duty. If a person is detected in the monitoring space, the character's line of sight is moved within a predetermined range, and the character's line of sight is moved within a predetermined range. Provided is a monitoring display device having a display control unit that controls the display of the movement of a security guard character by controlling the image display of the character so as to perform a tracking movement directed toward a person.

In this monitoring display device, it is preferable that the display control unit controls the image display of the character so that the motion range of the character during the tracking motion is larger than the motion range of the character during the watching motion.

In this monitoring display device, the display control unit sets a range of motion to move the character's line of sight by moving the character's eyes or face in the looking motion, and moves the line of sight by moving the character's entire body in the tracking motion. It is preferable to set the operating range as follows.

This surveillance display device further includes an imaging unit that captures an image of the surveillance space, and the detection unit detects a person from the image captured by the imaging unit and determines whether or not the detected person is performing a predetermined action. It is preferable that the display control unit controls the image display of the character so as to perform a tracking operation when the detected person is performing a predetermined action.

In this monitoring display device, when the display control unit continuously performs the tracking operation for a predetermined person in the monitoring space for a predetermined time or more, the display control unit terminates the tracking operation and performs a lookout action, and in the lookout action, When a predetermined person exists in the direction in which the character is looking, it is preferable to control the image display of the character so as to temporarily stop the movement of the character's line of sight.

In this monitoring display device, it is preferable that the display control section controls the image display of the character so as to change the direction of the character's line of sight according to the number of people detected by the detection section.

The surveillance display device according to the present invention can display an image that looks more like a real security guard when monitoring a surveillance space while displaying an image of a character imitating a security guard.

1 is a diagram showing the overall system configuration of a monitoring system 1. FIG. 1 is a schematic diagram showing the appearance of a monitoring display device 10. FIG. 3 is a flowchart illustrating an example of operation of display processing. (a) is a flowchart illustrating an example of the operation of image display processing related to the looking motion, and (b) is a flowchart illustrating an example of the operation of image display processing related to the tracking motion. (a) and (b) are schematic diagrams showing an example of a character's line of sight. (a) and (b) are schematic diagrams showing an example of a character's line of sight.

Hereinafter, a monitoring system according to an embodiment will be described with reference to the drawings.

FIG. 1 is a diagram showing the overall system configuration of a monitoring system 1 according to an embodiment. As shown in FIG. 1, the monitoring system 1 includes a plurality of monitoring display devices 10, a management device 20, and the like. Each monitoring display device 10 is installed in a different monitoring space in a monitoring target facility such as an airport, a train station, a commercial facility (department store, supermarket, etc.), a box office, a corporate office, etc., and monitors and guards each monitoring space. It is a terminal. The management device 20 is installed on a monitoring desk or the like of a security center operated by a security company or a disaster prevention center in a facility to be monitored, and is a device that collects and manages monitoring results (captured images, etc.) by each monitoring display device 10. . Each monitoring display device 10 and management device 20 are connected to each other via a communication network such as an intranet, the Internet, or a mobile phone network.
Note that the monitoring system 1 may include a plurality of management devices 20 that respectively manage a plurality of different monitoring targets. In that case, the monitoring system 1 further includes a server device located at a monitoring center installed at a location away from each monitoring target, and the server device collects and manages the information managed by each management device 20. It's okay.

The monitoring display device 10 is a display device such as a mirror signage. The monitoring display device 10 includes a display section 11, a sensor 12, an imaging section 13, an audio input section 14, an audio output section 15, a communication section 16, a storage section 17, a control section 18, and the like.

The display unit 11 is a mirror type display or the like, and displays various information such as images and text according to instructions from the control unit 18. In particular, the display unit 11 displays an image of a character (hereinafter sometimes referred to as a security guard character) imitating a security guard standing guard in accordance with instructions from the control unit 18 . Note that the display unit 11 may be a non-mirror type display such as a liquid crystal display or an organic EL display. Details of the display section 11 will be described later.

The sensor 12 is a human body detection sensor that periodically detects a person in the monitoring space and determines the position of the detected person. As the sensor 12, for example, a laser ranging sensor is used. A laser ranging sensor irradiates a near-infrared pulsed laser with a predetermined wavelength (e.g., approximately 870 nm) at predetermined angular steps (e.g., 0.25°) in the horizontal direction over a preset scanning range (e.g., 180°). Then, the reflected light of the laser is detected. A laser ranging sensor measures the distance to an object that reflects a laser beam, for example, using the Time-of-Flight method. The sensor 12 scans the entire scanning range at a fixed period (for example, 200 msec), measures the distance to the point where the laser is reflected in each direction within the scanning range, and uses the pulsed laser to calculate the measured distance. The distance measurement data associated with the direction in which the light is projected is output to the control unit 18. Note that the sensor 12 may determine the position of the person using other known ranging methods such as a phase difference method and a triangulation method. Further, a plurality of sensors 12 may be provided.

The imaging unit 13 includes a photoelectric conversion element that is sensitive to visible light, such as a CCD element or a C-MOS element, an imaging optical system that forms an image on the photoelectric conversion element, and an electric power output from the photoelectric conversion element. It has an A/D converter that amplifies the signal and performs analog/digital (A/D) conversion. The imaging unit 13 converts the photographed images of each RGB color into a digital input image in which each pixel has a luminance value in the range of 0 to 255, and outputs the digital input image to the control unit 18. Note that a plurality of imaging units 13 may be provided.
The audio input unit 14 includes a microphone and an A/D converter that amplifies the electrical signal output from the photoelectric conversion element and performs analog/digital (A/D) conversion. The audio input unit 14 converts the input audio into digital input audio and outputs it to the control unit 18 .
The audio output unit 15 is, for example, a speaker or the like, and outputs audio according to instructions from the control unit 18.

The communication unit 16 has a communication interface circuit compliant with, for example, TCP/IP, and is connected to a communication network such as an intranet or the Internet. Alternatively, the communication unit 16 has a communication interface circuit compliant with, for example, the W-CDMA system or the LTE system, and connects to a communication network such as a mobile communication network via a base station. The communication unit 16 outputs data received from the communication network to the control unit 18, and transmits data input from the control unit 18 to the communication network.

The storage unit 17 includes a semiconductor memory such as ROM or RAM, a magnetic disk or an optical disk drive such as CD-ROM or DVD-ROM, and its recording medium. The storage unit 17 stores computer programs and various data for controlling the monitoring display device 10, and inputs and outputs this information to and from the control unit 18. The computer program may be installed in the storage unit 17 from a computer-readable portable recording medium such as a CD-ROM or DVD-ROM using a known setup program or the like.
The storage unit 17 also stores, as data, the installation position of the monitoring display device 10 in the monitoring space and the position of specific areas such as entrance gates and restricted areas. Furthermore, the storage unit 17 stores the arrangement position of the sensor 12 in the monitoring space and the irradiation direction of the pulsed laser as data. The storage unit 17 also stores, as data, map information indicating the correspondence between each pixel in the image captured by the imaging unit 13 and each position in the monitoring space.

The control unit 18 includes a processor such as a CPU or an MPU, a memory such as a ROM or a RAM, and peripheral circuits thereof, and executes various signal processing of the monitoring display device 10. The control unit 18 includes a detection unit 181, a display control unit 182, etc., which are implemented as functional modules of a program running on a processor. The display control unit 182 displays and controls the motion of the displayed character according to the recognized situation of the monitoring space. Furthermore, when the display control unit 182 detects a person who has stopped near the monitoring display device 10 (for example, within 1 m) or detects an utterance from the person, the display control unit 182 controls the audio output so as to interact with the person. . Note that as the control unit 18, a DSP, LSI, ASIC, FPGA, etc. may be used.
The control unit 18 transmits the input image captured by the imaging unit 13 of the monitoring space or the input sound collected by the audio input unit 14 to the management device 20 via the communication unit 16. Further, the control unit 18 outputs the voice of the monitor received from the management device 20 via the communication unit 16 from the voice output unit 15.

The management device 20 is, for example, a personal computer. The management device 20 includes a second display section 21, a second audio input section 22, a second audio output section 23, a second communication section 24, a second storage section 25, a second control section 26, and the like.

The second display section 21 is a liquid crystal display, an organic EL display, or the like, and displays various information such as images and text according to instructions from the second control section 26.
The second audio input unit 22 has the same configuration as the audio input unit 14, converts the input audio into digital audio data, and outputs the digital audio data to the second control unit 26.
The second audio output unit 23 has the same configuration as the audio output unit 15 and outputs audio according to instructions from the second control unit 26.

The second communication unit 24 has a communication interface circuit compliant with, for example, TCP/IP, and is connected to a communication network such as an intranet or the Internet. The second communication unit 24 outputs data received from the communication network to the second control unit 26, and transmits data input from the second control unit 26 to the communication network.

The second storage unit 25 includes a semiconductor memory, a magnetic disk or an optical disk drive, and its recording medium, similar to the storage unit 17. The second storage unit 25 stores computer programs and various data for controlling the management device 20, and inputs and outputs this information to and from the second control unit 26. The computer program may be installed in the second storage unit 25 from a computer-readable portable recording medium such as a CD-ROM or DVD-ROM using a known setup program or the like.

The second control unit 26 has a processor, memory, and peripheral circuits similar to those of the control unit 18, and executes various signal processing of the management device 20. Note that a DSP, LSI, ASIC, FPGA, etc. may be used as the second control unit 26.
The second control unit 26 displays the input image received from the monitoring display device 10 via the second communication unit 24 on the second display unit 21, and also displays the input image received from the monitoring display device 10 via the second communication unit 24. The input audio is output from the second audio output section 23. Further, the second control unit 26 transmits the voice input by the monitor from the second voice input unit 22 to the monitoring display device 10 via the second communication unit 24.

FIG. 2 is a schematic diagram showing the appearance of the monitoring display device 10. As shown in FIG. As shown in FIG. 2, the display section 11 of the monitoring display device 10 is supported by a pedestal 19. The monitoring display device 10 is arranged so that the display section 11 faces the monitoring space A.
The display unit 11 is formed, for example, by bonding a mirror panel 111 to the display surface of a monitor such as a liquid crystal display or an organic EL display. The mirror panel 111 reflects light from the outside (monitoring space side) and transmits light (image) from the inside (monitor side). Thereby, the display unit 11 displays the image of the security guard character C output from the control unit 18 while reflecting and displaying the images of the persons H1 and H2 passing back and forth in the monitoring space A.
The sensor 12 is arranged on the outer frame 112 of the mirror panel 111 so as to be able to project near-infrared pulsed laser toward the monitoring space A. Similarly, in the outer frame 112, an imaging unit 13 is arranged to be able to image the monitoring space A, an audio input unit 14 is arranged to be able to collect the sound of the monitoring space A, and an audio output unit 15 is arranged to be able to capture the sound of the monitoring space A. It is arranged so that it can output sound toward the target. Note that the arrangement of the sensor 12, the imaging section 13, the audio input section 14, and the audio output section 15 is not limited to the outer frame 112, and may be arranged beyond the mirror panel 111.

FIG. 3 is a flowchart illustrating an example of the operation of display processing by the monitoring display device 10. This flowchart is executed mainly by the control unit 18 in cooperation with each element of the monitoring display device 10 based on a program stored in the storage unit 17 in advance. While the display process is being executed, the display control unit 182 continues to display the image of the security guard character on the display unit 11.

First, the detection unit 181 causes the imaging unit 13 to image the monitoring space, and acquires an input image of the monitoring space from the imaging unit 13 (step S101).

Next, the detection unit 181 executes detection processing to detect a person in the monitoring space (step S102).
The detection unit 181 detects a person from an input image. The detection unit 181 uses, for example, inter-frame difference processing or background difference processing to extract a changing region having a temporal change in luminance value in a plurality of sequentially photographed input images. The detection unit 181 extracts a changed region that is considered to be a person as a human region based on the feature values such as the size and shape of each extracted changed region. The detection unit 181 extracts edge pixels from the extracted human region using a Sobel filter or the like so that the direction of inclination of the luminance change can be determined. The detection unit 181 detects an elliptical edge distribution having a predetermined size that approximates the contour shape of the head from the extracted edge pixels using, for example, generalized Hough transform, and detects an edge distribution surrounded by the edge distribution. The area is extracted as a face area. Note that the detection unit 181 may extract the face area using other known methods such as an Adaboost classifier. When a face area is extracted from an input image, the detection unit 181 detects a person included in a person area corresponding to the face area as a person in the monitoring space. Then, based on the map information, the detection unit 181 identifies the position in the monitoring space corresponding to the lowest pixel of the person area, where the feet of the detected person are captured, as the person's position. When a plurality of face areas are extracted, the detection unit 181 detects a person for each extracted face area and specifies the person's position.

Note that the detection unit 181 may acquire distance measurement data from the sensor 12 and detect a person from the acquired distance measurement data. The detection unit 181 compares the acquired distance measurement data with preset reference measurement data, extracts a direction (scanning direction) in which a distance change of more than a predetermined value occurs, and detects continuity of more than a predetermined value (for example, a general A distance change direction group having a human size) is detected as a person in the monitoring space. The detection unit 181 detects the monitoring based on the arrangement position of the sensor 12 in the monitoring space, the irradiation direction of the pulsed laser, and representative distance measurement data related to the detected person (for example, the scanning direction and measured distance of the center of the distance change direction group). Identify a person's position in space. When a plurality of distance change direction groups are extracted, the detection unit 181 determines whether or not each distance change direction group is a person, and specifies the person's position.
Further, the detection unit 181 may acquire input voice from the voice input unit 14 and detect a person from the acquired input voice. The detection unit 181 determines that a person is present in the monitoring space when the sound pressure of the input voice is equal to or higher than a preset threshold. Note that the detection unit 181 may determine that a person is present in the monitoring space when the frequency of the input voice is within a preset range.

Next, the detection unit 181 determines whether a person is detected in the detection process (step S103).

If no person is detected in the detection process, the display control unit 182 controls the image display of the security guard character to perform a watching action (step S104), and returns the process to step S101. The details of the watching operation will be described later.

On the other hand, if a person is detected in the detection process, the detection unit 181 executes tracking process of the detected person (step S105).
The detection unit 181 performs tracking processing on the human regions extracted across a plurality of input images using a known tracking technique, and tracks the human regions by associating human regions in which the same person is photographed. For example, the detection unit 181 calculates the distance between the centroid position of the human area extracted from the current frame and the centroid position of the human area extracted from the previous frame, and if the distance is less than a threshold value, detects the human area. The people in the images are associated as the same person. Note that when a plurality of human regions are extracted, the detection unit 181 associates the persons appearing in the human region whose center of gravity is closest among the human regions whose distance is less than or equal to the threshold value as the same person. Furthermore, the detection unit 181 may perform tracking processing of the human region using other known methods such as optical flow and particle filtering.
Note that when the detection unit 181 detects a person from the ranging data acquired from the sensor 12, it calculates the distance between the currently detected person position and the previously detected person position, and if the distance is less than a threshold value, The person corresponding to the person's position is associated with the person as the same person. Note that when a plurality of people are detected, the detection unit 181 associates the person corresponding to the closest person position among the person positions whose distance is less than or equal to the threshold value as the same person.

Next, the detection unit 181 determines, for each person associated in the tracking process, whether each detected person is performing a predetermined action (step S106). The predetermined action is a new entry into the surveillance space or an approach to a specific area. If the currently detected person is not associated with any of the previously detected persons, the detection unit 181 determines that the currently detected person has newly entered the monitoring space, and performs a predetermined action. It is determined that the Further, when the position of the person in the monitoring space is within a predetermined distance from the position of the preset specific area, the detection unit 181 determines that the person is approaching the specific area, and performs the predetermined action. It is determined that there is. In other cases, the detection unit 181 determines that each person is not performing the predetermined action. Note that the predetermined behavior is not limited to the above example, and any behavior that requires more attention (for example, wandering around in the monitoring space, making a loud noise, etc.) can be set as the predetermined behavior.

If none of the detected persons is performing the predetermined action, the display control unit 182 controls the image display of the security guard character to perform a watching action (step S104), and returns the process to step S101. Return to.

On the other hand, if any of the detected persons is performing a predetermined action, the display control unit 182 controls the image display of the security guard character to perform a tracking action (step S107), and performs the processing. returns to step S101. Details of the tracking operation will be described later.

Note that the processes in steps S105 and S106 are omitted, and the display control unit 182 performs a watching operation when a person is not detected within the monitoring space, and performs a tracking operation when a person is detected within the monitoring space. The image display of the security guard character may be controlled so as to execute the process. In this case as well, the display control unit 182 can display an image that looks like a real security guard.
On the other hand, the display control unit 182 controls the image display of the security guard character so that the tracking operation is performed only when the detected person is performing a predetermined action. It becomes possible to make the security guard character have a strong presence.

Further, the display control unit 182 may control the image display of the security guard character so that the tracking operation is performed only when the detected person has a predetermined attribute. In that case, the detection unit 181 specifies the attributes of each person for each person associated in the tracking process. The detection unit 181 identifies age, gender, facial expression, etc. as attributes of the person.
For example, the detection unit 181 uses machine learning technology to identify attributes of a person. The monitoring display device 10 uses a plurality of sample images showing people with various attributes, and is configured in advance to output the attributes of the person shown in the sample image when each sample image is input. The learned discriminator is stored in the storage unit 17. The detection unit 181 cuts out a person area in the input image and inputs it to a classifier, and acquires the attribute output from the classifier as the attribute of the person in the person area.
Note that the detection unit 181 may identify the attributes of each person using a feature value matching technique. The monitoring and display device 10 associates feature amounts of each part such as "eyes,""nose," and "mouth" calculated from a plurality of sample images of people with various attributes with their attributes, and stores them in advance in a storage unit. Remember it on 17th. The feature amount is a Haar-Like feature amount, a HOG (Histogram of Oriented Gradient) feature amount, or the like. The detection unit 181 detects each part from the human region in the input image based on the characteristics such as the position or shape of each part, and calculates the feature amount. The detection unit 181 calculates the degree of similarity between the calculated feature quantity and each feature quantity stored in the storage unit 17, and selects the attribute associated with the feature quantity with the highest degree of similarity to the person area. Obtained as an attribute of the person in the image. The degree of similarity is a normalized cross-correlation value, etc.
The display control unit 182 controls the image display of the security guard character so that the tracking operation is performed only when the detected person has predetermined attributes (for example, male, 20 years or older, tense expression, etc.). Thereby, the display control unit 182 can make the security guard character have a strong presence with respect to the person who should pay more attention.

FIG. 4(a) is a flowchart illustrating an example of the operation of image display processing related to the watching operation. Note that the image display process shown in FIG. 4(a) is executed in step S104 of FIG. 3. Further, as will be described later, when the tracking operation is continuously performed for a predetermined time or more on a predetermined person, the display control unit 182 ends the tracking action and performs the keeping watch operation as shown in FIG. 4(a). Executes image display processing.

First, the display control unit 182 determines that there is a predetermined person who has been continuously detected as a person performing a predetermined action for a predetermined period of time or more in the line of sight direction of the currently displayed security guard character. It is determined whether or not to do so (step S201). For example, the monitoring display device 10 sets in advance, in the storage unit 17, a position in the monitoring space corresponding to each line-of-sight direction for each of a plurality of line-of-sight directions. If the position corresponding to the line-of-sight direction of the security guard character currently displayed overlaps with the latest person position of the predetermined person, the display control unit 182 determines that the predetermined person is present in the line-of-sight direction.

If the predetermined person does not exist in the line-of-sight direction of the security guard character, the display control unit 182 determines the line-of-sight direction of the guard character so that the line of sight of the guard character moves within a predetermined range (step S202).

FIGS. 5A and 5B are schematic diagrams illustrating an example of the line of sight of the security guard character during the watching motion. FIG. 5(a) shows the security guard character C in a state where the line of sight is directed to the rightmost side when viewed from the monitoring display device 10 side during the watching action, and FIG. The security guard character C is shown facing the leftmost side when viewed from the 10th side. In the watching motion, the line of sight of the security guard character C initially faces the front as shown in FIG. 2, and gradually moves to the right as time passes. When the line of sight of the security guard character C reaches the position shown in FIG. 5(a), it gradually moves to the left, and when it reaches the position shown in FIG. 5(b), it moves to the right again. . After this, this operation is repeated. The display control unit 182 determines the line of sight direction of the guard character C so that the line of sight of the guard character C changes from the previous state within a predetermined range. Note that the direction of movement of the line of sight of the security guard character C is not limited to the left-right direction (horizontal direction), but may be the up-down direction (vertical direction) or the diagonal direction. In addition, in the watching motion, the direction of the line of sight of the security guard character C may be set so that it does not always change from the front, or the movement of the line of sight may be temporarily stopped (for example, for 10 seconds) when the line of sight is facing the front. It's okay.

On the other hand, if the predetermined person is present in the line-of-sight direction of the security guard character, the display control unit 182 determines the line-of-sight direction of the security guard character so as to temporarily stop the movement of the line of sight (step S203). The display control unit 182 determines the line-of-sight direction of the security guard character so as to temporarily stop movement of the line-of-sight for a predetermined period of time (for example, 3 seconds). Thereby, the display control unit 182 can prevent a person who is present in the monitoring space for a long time from feeling uncomfortable due to being continuously watched by the security guard character. Furthermore, the display control unit 182 can cause the security guard character to gaze at the person in a natural manner, giving the security guard character the feeling that he or she is being watched (intimidation effect).

Next, the display control unit 182 updates the image display of the security guard character displayed on the display unit 11 so that it faces the line of sight direction determined in step S202 or S203 (step S204), and ends the series of steps. . For example, the monitoring display device 10 divides an image of a security guard character into predetermined parts such as eyes, nose, mouth, facial outline, neck, body, arms, legs, etc., and assigns polygons to each part. I'll keep it. Furthermore, the monitoring display device 10 sets in advance in the storage unit 17 the position and orientation of the polygon of each part corresponding to each viewing direction for each of the viewing directions. The display control unit 182 updates the image display of the security guard character by moving the polygon and moving each part in accordance with the polygon so that the security guard character faces the determined line of sight direction. Note that the monitoring display device 10 may store images of separate security guard characters in advance for each of a plurality of line-of-sight directions. In that case, the display control unit 182 updates the image display of the security guard character by displaying an image of the security guard character according to the viewing direction.
Furthermore, in the examples shown in FIGS. 5A and 5B, the display control unit 182 changes the line of sight by changing the direction of the eyes and the direction of the face of the security guard character. The line of sight may be changed by changing one or more of the orientation of the face (head) or the orientation of the entire body (whole body).

FIG. 4(b) is a flowchart illustrating an example of the operation of image display processing related to the tracking operation. Note that the image display process shown in FIG. 4(b) is executed in step S107 of FIG.

First, the display control unit 182 determines whether the tracking time of each person detected as a person performing a predetermined action in the monitoring space is longer than a predetermined time (step S301). The display control unit 182 displays, for each detected person, the time during which the security guard character is directing his/her line of sight (for example, the time during which the security guard character is performing a tracking operation, or the time from when each person enters the surveillance space to the current time). time) is calculated as the tracking time, and it is determined whether the tracking time of each person is longer than a predetermined time. The predetermined time is generally set to a time that makes the user aware that he/she is being watched by another person (security guard character C), and is set to, for example, 10 seconds. Further, the predetermined time may be set in consideration of a time period during which the user does not feel uncomfortable even if the user continues to be viewed by others. Note that the display control unit 182 determines whether or not the security guard character has been continuously detected for a predetermined time or longer in the surveillance space, instead of determining whether the security guard character is looking at the guard character for a predetermined time or longer. You may.

If the tracking time of all the persons detected as performing the predetermined action is longer than the predetermined time, the display control unit 182 ends the tracking operation and executes the keeping watch operation (step S302).

On the other hand, if the tracking time of any person detected as a person performing a predetermined action is not longer than the predetermined time, the display control unit 182 changes the line of sight of the security guard character to direct the line of sight to the detected person. The direction is determined (step S303). In particular, the display control unit 182 determines the line of sight direction of the security guard character so that the line of sight is directed toward a person whose tracking time is not longer than a predetermined time. The display control unit 182 determines the direction from the installation position of the monitoring display device 10 toward the position of the person detected by the detection unit 181 as the line-of-sight direction.

FIGS. 6A and 6B are schematic diagrams showing an example of the line of sight of the security guard character during the tracking operation. FIG. 6A shows the security guard character C in a state of directing his/her line of sight to the person H1 located on the right side when viewed from the monitoring display device 10 side during the tracking operation. On the other hand, FIG. 6(b) shows the security guard character C in a state of directing his/her line of sight to the person H2 located on the left side when viewed from the monitoring display device 10 side during the tracking operation. In the tracking action, the line of sight of the security guard character C is directed toward any person who has not performed the tracking action for a predetermined period of time or more among the persons detected as performing the predetermined action. If there are a plurality of corresponding persons, the person to whom the guard character C looks is changed at regular intervals. In this example, the display control unit 182 determines the line-of-sight direction so that the line-of-sight of the security guard character C is directed alternately toward the persons H1 and H2. Furthermore, the display control unit 182 makes the range in which the line of sight of the security guard character C can move in the tracking action larger than the predetermined range that is the range in which the line of sight of the guard character C in the watching action. For example, in the watching motion, the movable range is set to 45 degrees to the left and right based on the front of the security guard character C, and in the tracking motion, the movable range is set to 75 degrees to the left and right based on the front of the guard character C. . In this way, the movable range is set narrow in the looking motion, so the line of sight is moved by eye movement and/or facial movement, and in the tracking motion, the movable range is set wide, so the line of sight is moved by the movement of the eyes and/or the face. The line of sight is moved by adding the movement of . The reason why the movable range is set narrow in the lookout action is to monitor the surveillance space by the natural line of sight movement of the security guard character C, and the movable range is set wide in the tracking action. This is to give the security guard character C a sense of presence by allowing him or her to continuously look at the person. Thereby, the display control unit 182 can give a sense of presence to a person located at an arbitrary position within the surveillance space while directing the line of sight to the person present at an arbitrary position within the monitoring space.

Note that the display control unit 182 may change the direction in which the security guard character looks, depending on the number of people detected by the detection unit 181, particularly the number of people performing a predetermined action. For example, the display control unit 182 shortens the time that the security guard character C looks at each person as the number of people increases. Thereby, the display control unit 182 can efficiently give each person the feeling that they are being watched.
Further, the display control unit 182 may set a priority for each person. For example, the display control unit 182 sets the priority so that the closer the person is to the installation location of the monitoring display device 10 or the specific area, the higher the priority. Alternatively, the display control unit 182 may set the priority so that the higher the movement (movement distance) of a person, the higher the priority. Then, the display control unit 182 determines the line-of-sight direction so that the line-of-sight of the security guard character C is directed in descending order of priority. Note that the display control unit 182 may set a person with a higher priority to have a longer time to direct the line of sight of the security guard character C. Thereby, the display control unit 182 can give a person who is at higher risk a strong sense of being watched.
Further, the display control unit 182 may group each person based on the position of each person and/or the moving direction of each person. For example, the display control unit 182 may group one or more persons whose positions are within a specific distance, and/or one or more persons whose movement directions (vectors) differ within a specific range. summarized in. The display control unit 182 changes the group to which the security guard character C directs the line of sight at regular intervals, and changes the line of sight direction so that the line of sight of the security guard character C is directed to the center of gravity of the person position of each person included in the group. Determine. Thereby, the display control unit 182 can efficiently change the line of sight when there are many detected people.

Next, the display control unit 182 updates the image display of the security guard character displayed on the display unit 11 so as to face the determined line of sight direction in the same manner as the looking motion (step S304), and performs a series of steps. finish.
Note that in the examples shown in FIGS. 6A and 6B, the display control unit 182 changes the line of sight by changing the direction of the face of the security guard character, but the display control unit 182 changes the direction of the eyes or the direction of the entire body. The line of sight may be changed by changing.
The display control unit 182 also controls the security guard character so that the body parts (eyes, face, whole body, etc.) that the security guard character moves in the tracking motion, that is, the motion range is larger than the motion range of the security guard character in the watching motion. The image display of the character may also be controlled. For example, in order to change the line of sight of the security guard character, the display control unit 182 changes only the direction of the eyes in the watching motion, and changes the direction of the entire face or body in the tracking motion. Alternatively, the display control unit 182 changes the direction of the face in the looking motion, and changes the direction of the entire body in the tracking motion. That is, the display control unit 182 sets the range of motion to move the line of sight by moving the eyes or face of the security guard character in the watching motion, and moves the line of sight by moving the security guard character's entire body in the tracking motion. Set the operating range so that Thereby, the display control unit 182 can give the person a stronger sense of being watched. Further, even when the security guard character C faces the same direction in the watching action and the tracking action, the direction of the line of sight may be made different. For example, when directing the line of sight in a direction of 30 degrees to the left from the front, the line of sight is directed by the movement of the eyes in the looking motion, and the line of sight is directed by the movement of the entire body in the tracking motion. In addition, in the tracking motion, within the movable range of the looking motion, the line of sight is directed by the same movement (e.g., eye movement) as the looking motion, and when the range of motion exceeds the movable range of the looking motion, the movement range is enlarged ( For example, it may be possible to direct the gaze (movement of the entire body).
The display control unit 182 also controls the image display of the security guard character so that the movement range of the security guard character during the tracking operation is changed according to the distance from the installation position of the monitoring display device 10 to the person's position. Good too. For example, if the distance from the installation position of the monitoring display device 10 to the person's position is equal to or greater than the distance threshold, the display control unit 182 changes only the direction of the eyes, and changes the distance from the installation position of the monitoring display device 10 to the person's position. is less than the distance threshold, change the orientation of the entire face or body. This allows the display control unit 182 to more naturally direct the security guard character's line of sight toward the detected person.

Note that steps S201 and 203 in FIG. 4(a) and steps S301 and 302 in FIG. 4(b) are omitted, and when a person is detected in the monitoring space, the display control unit 182 The tracking operation may be performed regardless.

Further, in parallel with the display processing described above, the display control unit 182 causes the security guard character to perform a predetermined movement or utterance according to the situation of the monitoring space or according to a predetermined rule. For example, if the amount of change in the person's position detected by the detection unit 181 within a predetermined period is less than or equal to a threshold, the display control unit 182 determines that the person is standing still, and instructs the security guard character to make a predetermined movement or utter a voice. Let it run. Alternatively, the display control unit 182 applies known voice recognition technology and natural language processing technology to the input voice acquired from the voice input unit 14 to detect the content of the utterance by a person in the monitoring space, and applies the content of the utterance to the detected utterance content. If a predetermined keyword is included, the security guard character is caused to perform a predetermined movement or utterance. Alternatively, the display control unit 182 causes the security guard character to perform a predetermined movement or utterance every time a certain period of time elapses. For example, the display control unit 182 controls the image display of the security guard character displayed on the display unit 11 so that the security guard character takes a predetermined posture, such as a salute, as a predetermined movement. Furthermore, the display control unit 182 outputs a predetermined utterance from the audio output unit 11, such as a greeting such as "Hello" or a safety question such as "Are you okay?".

As described above, the monitoring display device 10 according to the present invention performs a monitoring operation when a person is not detected in the monitoring space, and performs a tracking operation when a person is detected. By controlling the image display of the security guard character, the display of the movement of the security guard character is controlled. As a result, when monitoring the surveillance space while displaying the image of the security guard character, the surveillance display device 10 allows the security guard character to behave naturally to demonstrate its presence as a security guard, thereby creating a more authentic image. It becomes possible to display an image that looks like a security guard. Therefore, the monitoring display device 10 can give a person present in the monitoring space a sense of being watched and a sense of security that they are being watched over.

10 monitoring display device, 11 display unit, 13 imaging unit, 181 detection unit, 182 display control unit

Claims (10)

a display section that displays an image of the character;
a detection unit that detects a plurality of people in a surveillance space and detects a person of interest from among the detected people ;
A watching action is performed in which the line of sight of the character is moved within a predetermined range, and when a person of interest is detected in the monitoring space, the line of sight of the character is moved to the one of the detected object of interest among the plurality of people. a display control unit that controls the display of the character's motion by controlling the image display of the character to switch to and execute a tracking motion directed toward a person;
A monitoring display device comprising: The surveillance display device according to claim 1, wherein the character is a character imitating a person standing guard. The display control unit executes the watching operation when a person is not detected in the monitoring space, and executes the watching operation when a person is detected in the monitoring space when the person is the person of interest. The monitoring display device according to claim 1 or 2, which performs the tracking operation. When a person is detected in the monitoring space, the display control unit performs a tracking operation on the person of interest when the detected person includes the person of interest, and The monitoring display device according to any one of claims 1 to 3, wherein the monitoring and displaying device performs the watching operation when the person of interest is not included in the people who have been looked at. further comprising an imaging unit that captures an image of the monitoring space,
The detection unit detects a person from the image captured by the imaging unit, and also specifies an attribute of the detected person, or determines whether the detected person is performing a predetermined action,
If the detected person has a predetermined attribute or is performing the predetermined action, the display control unit executes the tracking operation with the detected person as the person of interest. The monitoring display device according to any one of the above. The display control unit displays an image of the character so that the line of sight of the character moves within a predetermined range in the looking motion, and the movable range of the line of sight of the character is larger than the predetermined range in the tracking motion. The monitoring and display device according to any one of claims 1 to 5, which controls the monitoring and display device. Any one of claims 1 to 6 , wherein the display control unit controls image display of the character so that a movement range of the character in the tracking movement is larger than a movement range of the character in the looking movement. The monitoring display device described in Section 1. The display control unit sets the motion range so that the line of sight is moved by moving the character's eyes or face in the looking motion, and moves the line of sight by moving the character's entire body in the tracking motion. The monitoring and display device according to claim 7 , wherein the operating range is set as follows. a display section that displays an image of the character;
a detection unit that detects a person in the surveillance space;
Execute a watching action to move the line of sight of the character within a predetermined range, and when a person of interest is detected in the monitoring space, perform a tracking operation of directing the line of sight of the character to the detected person of interest. a display control unit that controls the display of the character's actions by controlling the image display of the character to be switched and executed;
When the display control unit continuously executes the tracking operation for a predetermined time period or longer with respect to the person of interest in the monitoring space, the display control unit terminates the tracking operation and executes the looking operation, and performs the looking operation. controlling the display of the image of the character so as to temporarily stop movement of the line of sight when the person of interest is present in the direction in which the character is directing his/her line of sight;
A monitoring display device characterized by: a display section that displays an image of the character;
a detection unit that detects a person in the surveillance space;
Execute a watching action to move the line of sight of the character within a predetermined range, and when a person of interest is detected in the monitoring space, perform a tracking operation of directing the line of sight of the character to the detected person of interest. a display control unit that controls the display of the character's actions by controlling the image display of the character to be switched and executed;
The display control unit controls the image display of the character so as to change the direction of the character's line of sight according to the number of people detected by the detection unit.
A monitoring display device characterized by:

Images