JP2020182092A - Security system and monitoring display - Google Patents

Abstract

To provide a security system or the like that enables a security guard who is an operator to easily visually recognize the response status in each monitoring space.SOLUTION: A security system according to the present invention includes a plurality of monitor display devices, and a management device that is communicatively connected to the plurality of monitor display devices, and each of the monitor display devices displays an image of a character imitating a standing guard, images a monitoring space, detects a person in the monitoring space, and makes the character interact with the person to perform first response, specifies a dialogue status of a dialogue, associates the captured image with the dialogue status, and transmits the captured image and the dialogue status to the management device, and the management device displays the dialogue status and the image received from each of the monitor display devices in association with each other.SELECTED DRAWING: Figure 1

Description

The present invention relates to a security system and a monitoring display device.

Conventionally, security guards have been assigned to facilities such as transportation facilities such as airports and stations, commercial facilities, entertainment facilities, and corporate offices to perform guards and patrols. In addition to deterring crimes by the threatening effect brought about by surveillance, outpost security in such facilities has the aspect of improving services in the facilities by providing guidance and other services to facility users. Have.

On the other hand, due to the shortage of security guards, a plurality of surveillance cameras are installed in the facility instead of allocating security guards. In this case, the guards resident in the monitoring room inside and outside the facility monitor the monitoring space inside the facility by checking the images of a plurality of surveillance cameras. Patent Document 1 describes a surveillance system that improves surveillance efficiency by determining the importance of a surveillance camera based on the installation location and displaying video data of the surveillance camera according to the importance.

Japanese Unexamined Patent Publication No. 2008-278368

By the way, as described above, the guards also have an aspect of providing guidance to the users of the facility, but the surveillance camera cannot substitute for such a response work. Therefore, the introduction of a monitoring display device having an automatic response function for users is being considered. When such a monitoring display device is introduced, when it becomes difficult for the monitoring display device to respond, the operator will respond to the user directly by a remote call or the like or toward the monitoring space. Therefore, the monitoring display device is required to allow the operator to easily grasp the situation of each monitoring space so that the operator can respond smoothly.

The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a security system and a monitoring display device that enable an operator to easily grasp the situation of each monitoring space.

The security system according to the present invention is a security system having a plurality of monitoring and display devices and a management device communicably connected to the plurality of monitoring and displaying devices, and each of the plurality of monitoring and displaying devices stands by. A first display unit that displays an image of a character imitating a security guard, an audio input / output unit that outputs the character's voice and inputs surrounding voice, an imaging unit that images the monitoring space, and a monitoring space An image pickup includes a detection unit that detects a person, a first response unit that performs the first response by having the character interact with the detected person via a voice input / output unit, and a specific unit that specifies the dialogue status of the dialogue. A transmission unit that associates an image captured by the unit with a dialogue status and transmits the dialogue status to the management device, and the management device displays the image received from each monitoring display device and the dialogue status in association with each other. It is characterized by having two display units.

Further, in the security system according to the present invention, it is preferable that the specific unit specifies whether or not the character is in a dialogue with the person whose character is detected as the dialogue situation.

Further, in the security system according to the present invention, each of the plurality of monitoring display devices further includes a determination unit for determining whether or not a predetermined switching condition regarding the detected movement of the person and the utterance or dialogue is satisfied, and transmits. When it is determined that the switching condition is satisfied, the unit sends a request for switching from the first response to the second response by a telephone call with the operator who operates the management device, and the management device displays the monitoring display device. The second display unit includes a call unit that enables a call with a person detected via the device and a second response unit that performs a switching process to the second response based on the received switching request. Is received, it is preferable to display the dialogue status in association with the image received from the monitoring display device that transmitted the switching request.

Further, in the security system according to the present invention, the transmission unit transmits the dialogue history indicating the dialogue content between the character and the detected person by the first response unit to the management device in association with the image, and the second display unit transmits the dialogue history. , It is preferable to display the dialogue history corresponding to the monitoring display device together with the image received from the monitoring display device.

Further, in the security system according to the present invention, the transmitting unit sets the urgency of the switching request, and the second display unit sets the display mode of the image received from the monitoring display device that transmitted the switching request to the urgent switching request. It is preferable to change it according to the degree.

The monitoring display device according to the present invention is an image in the monitoring space captured by each monitoring display device received from a plurality of monitoring display devices, and a character imitating a guard who is standing by each monitoring display device. A monitoring display device that is communicably connected to a management device that has a second display unit that displays the interaction status of the person in the monitoring space in association with each other, and is a character that imitates a guard who is standing by. A first display unit that displays an image, a voice input / output unit that outputs a character's voice and inputs surrounding voice, an imaging unit that images the monitoring space, and a detection unit that detects a person in the monitoring space. The first response unit that performs the first response by causing the character to interact with the detected person via the voice input / output unit, the specific unit that specifies the dialogue status of the dialogue, and the image captured by the imaging unit. It is characterized by including a transmission unit that associates with the dialogue status and transmits the transmission to the management device.

The security system and the monitoring display device according to the present invention enable the operator to easily grasp the situation of each monitoring space.

It is a figure which shows an example of the schematic structure of the security system 1. It is a figure which shows an example of the schematic structure of the monitoring display device 2. It is a schematic diagram for demonstrating an example of the appearance of the monitoring display device 2. It is a figure which shows an example of the schematic structure of the management device 3. It is a figure which shows an example of the data structure of the dialogue purpose table 261. It is a figure which shows an example of the data structure of the switching condition table 262. It is a sequence diagram which shows an example of the flow of the monitoring process by a security system 1. It is a flow chart which shows an example of the flow of the response processing by the security system 1. It is a figure which shows an example of the monitoring screen 900. It is a figure which shows an example of the monitoring screen 1000. It is a figure which shows an example of the reception screen 1100.

Hereinafter, various embodiments of the present invention will be described with reference to the drawings. However, it should be noted that the technical scope of the present invention is not limited to those embodiments, but extends to the inventions described in the claims and their equivalents.

FIG. 1 is a diagram showing an example of a schematic configuration of a security system 1 according to the present invention. The security system 1 is used, for example, to provide a security service including a monitoring function and a reception function in a facility to be monitored.

The security system 1 has a plurality of monitoring display devices 2a-2d (hereinafter, referred to as monitoring display devices 2 when it is not particularly necessary to distinguish them) and a management device 3. The monitoring display device 2 and the management device 3 are connected to each other so as to be able to communicate with each other via a network 4 such as the Internet or an intranet.

The monitoring display device 2 is installed in different monitoring spaces in the facility to be monitored, monitors each monitoring space, and provides guidance to the users of the facility.

The management device 3 is an information processing device such as a workstation or a PC. The management device 3 may be a multifunctional mobile phone, a tablet terminal, or the like capable of performing predetermined information processing. The management device 3 is installed in, for example, a monitoring room in the facility and is operated by an operator resident in the monitoring room. The management device 3 may be installed outside the facility.

The monitoring display device 2 displays a character (hereinafter, referred to as a “guard character”) that imitates a guard who is standing by. The monitoring display device 2 detects a person in the monitoring space such as a facility user. The monitoring display device 2 recognizes the movement and utterance of the detected person, and makes the guard character interact with the detected person to perform a response (hereinafter, referred to as "first response"). Further, the monitoring display device 2 has a video call function with the management device 3, and the operator who operates the management device 3 by transmitting a switching request to the management device 3 responds to the detected person (hereinafter, "" It is called "second response").

FIG. 2 is a diagram showing an example of a schematic configuration of the monitoring display device 2. The monitoring display device 2 includes a first display unit 21, a first imaging unit 22, a sensor 23, a first audio input / output unit 24, a first communication unit 25, a first storage unit 26, and a first control. A unit 27 is provided.

The first display unit 21 is a mirror type display, and displays various information such as images and texts according to the data supplied from the first control unit 27. In particular, the first display unit 21 displays the guard character according to the instruction from the first control unit 27. The first display unit 21 may be a non-mirror type display such as a liquid crystal display or an organic EL display.

The first image pickup unit 22 includes an imaging optical system, an image pickup element, an image processing unit, and the like. The imaging optical system is, for example, an optical lens, and a light flux from a subject is imaged on an imaging surface of an image pickup device. The image pickup device is, for example, a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like, and outputs an image signal of a subject image formed on the image pickup surface. The image processing unit generates image data in a predetermined file format from the image signal generated by the image sensor and supplies it to the first control unit 27.

The sensor 23 is a device that measures the distance to an object around the monitoring display device 2, and is, for example, a TOF (Time of Flight) type laser ranging sensor. The sensor 23 irradiates the pulse-modulated laser light in each direction within a predetermined scanning range, and calculates the distance to the reflection point in each direction based on the time until the reflected light is detected. Subsequently, the sensor 23 associates the direction in which the laser beam is irradiated with the distance to the reflection point in each direction and supplies the distance measurement data to the first control unit 27.

The first audio input / output unit 24 is, for example, a microphone and a speaker. The first audio input / output unit 24 converts the audio around the monitoring display device 2 into an electric signal and supplies it to the first control unit 27, and also converts the electric signal supplied from the first control unit 27 into an electric signal. And output.

The first communication unit 25 includes a communication interface circuit that enables the monitoring display device 2 to communicate with the management device 3 via the network 4. The communication interface circuit included in the first communication unit 25 is an LTE (Long Term Evolution) or a communication interface circuit of a wireless LAN or a wired LAN. The first communication unit 25 transmits the data supplied from the first control unit 27 to the management device 3, receives the data transmitted from the management device 3, and supplies the data to the first control unit 27.

The first storage unit 26 is a device for storing a program or data, and includes, for example, a semiconductor memory device. The first storage unit 26 stores an operating system program, a driver program, an application program, data, and the like used for processing by the first control unit 27. The program is installed in the first storage unit 26 from a computer-readable portable storage medium such as a CD-ROM or a DVD-ROM using a known setup program or the like.

In addition, the first storage unit 26 stores the dialogue purpose table 261 and the switching condition table 262 as data.

The first control unit 27 includes one or a plurality of processors and peripheral circuits thereof. The first control unit 27 is, for example, a CPU (Central Processing Unit), and controls the operation of the monitoring display device 2 in an integrated manner. The first control unit 27 may be a DSP (Digital Signal Processor), an LSI (Large-Scaled IC), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like. The first control unit 27 so that various processes of the monitoring display device 2 are executed in an appropriate procedure based on the program stored in the first storage unit 26 or the voice input from the first voice input / output unit 24. , Controls the operation of the first display unit 21 and the first communication unit 25. The first control unit 27 executes processing based on the program stored in the first storage unit 26. In addition, the first control unit 27 can execute a plurality of programs in parallel.

The first control unit 27 includes a first display control unit 271, a detection unit 272, a first response unit 273, a specific unit 274, a determination unit 275, and a transmission unit 276 as its functional blocks. The first display control unit 271 displays and controls the operation of the guard character to be displayed according to the recognized monitoring space status and the like. Further, the first response unit 273 controls the voice output so as to interact with the person when detecting a person who has stopped in the vicinity of the monitoring display device 10 (for example, within 1 m) or detecting an utterance from the person. To do. The first display control unit 271, the detection unit 272, the first response unit 273, the specific unit 274, the determination unit 275, and the transmission unit 276 are functional modules realized by a program executed by the first control unit 27. Further, each of these parts may be mounted on the monitoring display device 2 as firmware.

FIG. 3 is a schematic diagram for explaining the appearance of the monitoring display device 2. As shown in FIG. 3, the first display unit 21 of the monitoring display device 2 is supported by the pedestal 28. The monitoring display device 2 is arranged so that the first display unit 21 faces the monitoring space A.

The first display unit 21 is formed, for example, by attaching the mirror panel 211 to the display surface of the monitor. The mirror panel 211 transmits light (image) from the inside (monitor side) while reflecting light from the outside (monitoring space side). As a result, the first display unit 21 reflects and displays the images of the persons H1 and H2 coming and going in the monitoring space A, and displays the image of the guard character C output from the first control unit 27.

A sensor 23 is arranged on the outer frame 212 of the mirror panel 211 so as to be able to irradiate a near-infrared pulse laser toward the monitoring space A. Similarly, in the outer frame 212, the first imaging unit 22 is arranged so that the monitoring space A can be imaged, and the first audio input / output unit 24 can collect the sound of the monitoring space A and directs the sound to the monitoring space A. Arranged so that sound can be output. The arrangement of the sensor 23, the first imaging unit 22, and the first audio input / output unit 24 is not limited to the outer frame 212, and may be arranged through the mirror panel 211.

FIG. 4 is a diagram showing an example of a schematic configuration of the management device 3. The management device 3 includes a second communication unit 31, a second display unit 32, an operation unit 33, a second audio input / output unit 34, a second imaging unit 35, a second storage unit 36, and a second control. A unit 37 is provided.

The configurations of the second communication unit 31, the second audio input / output unit 34, the second imaging unit 35, and the second storage unit 36 are the first communication unit 25, the first audio input / output unit 24, the first imaging unit 22, and the second storage unit 36, respectively. This is the same as the first storage unit 26.

The second display unit 32 is a device that displays an image or the like to the operator, and is, for example, a liquid crystal display or an organic EL display. The second display unit 32 displays an image or the like according to the data supplied from the second control unit 37.

The operation unit 33 is a device for receiving the operation of the management device 3 by the operator, and is, for example, a keyboard, a mouse, or the like. The operation unit 33 accepts input operations, selection operations, and the like of characters, numbers, symbols, etc. by the user. When the operation unit 33 receives an operation by the user, the operation unit 33 generates a signal corresponding to the operation and supplies the signal to the second control unit 37.

The second control unit 37 includes one or more processors and peripheral circuits thereof. The second control unit 37 is, for example, a CPU, and controls the operation of the management device 3 in an integrated manner. The second control unit 37 may be a DSP, LSI, ASIC, FPGA, or the like. The second control unit 37 operates the second communication unit 31 and the second display unit 32 so that various processes of the management device 3 are executed in an appropriate procedure based on the program stored in the management device 3. Control. The second control unit 37 executes the process based on the program stored in the management device 3. In addition, the second control unit 37 can execute a plurality of programs in parallel.

The second control unit 37 includes a second display control unit 371 and a second response unit 372 as its functional blocks. Each of these units is a functional module realized by a program executed by the second control unit 37. Further, each of these parts may be mounted on the management device 3 as firmware.

FIG. 5 is a diagram showing an example of the data structure of the dialogue purpose table 261 stored in the first storage unit 26 of the monitoring display device 2. The dialogue purpose table 261 stores a plurality of dialogue conditions, the dialogue purpose, and the utterance content in association with each other.

The dialogue condition is a condition for the monitoring display device 2 to have a dialogue in the first response, and is a condition related to the movement or utterance of the person detected by the monitoring display device 2. The purpose of the dialogue is the purpose of the dialogue that takes place when the associated dialogue conditions are met. The purpose of the dialogue includes "greeting", "guidance", "safety question" and the like. The utterance content is the content uttered (that is, output as voice) by the guard character when the associated dialogue condition is satisfied, and the content according to the associated dialogue purpose is stored.

In the example shown in FIG. 5, speech content that dialogue objects and referred to as "greeting" dialogue condition "person older than 5 seconds close", "Hello." Is associated with. That is, when the monitoring display device 2 determines that the dialogue condition that "a person approaches and 5 seconds or more have passed" is satisfied, the dialogue for the purpose of "greeting" is started in the first response. Then, the monitoring display device 2, to the speech as "Hello." Security guards character. When the position of the person in the monitoring space is within a predetermined distance from the position of the specific area set in advance, the detection unit 272 determines that the person is approaching the specific area.

Similarly, when the monitoring display device 2 determines that the dialogue condition of "questioning the place from a person" is satisfied, the dialogue for the purpose of dialogue is started in the first response. Then, the monitoring display device 2 acquires the route to the place where the person asked, from the first storage unit 26, and tells the guard character, "Please proceed to the passage on your right to XX." , Speak to guide the acquired route.

The various data in the dialogue purpose table 261 are appropriately set in advance by the security service provider and stored in the first storage unit 26.

FIG. 6 is an example of the data structure of the switching condition table 262 stored in the first storage unit 26. The switching condition table 262 stores the switching condition, the dialogue continuation information, the dialogue purpose, and the urgency in association with each other.

The switching condition is a condition for the monitoring display device 2 to transmit a switching request from the first response to the second response to the management device, and the movement and utterance of the person detected by the monitoring display device 2 or the detected person. It is a condition regarding dialogue in the first response of. The dialogue continuation information is information indicating whether or not the dialogue is continuing in the first response, "during dialogue" indicates that the dialogue is continuing, and "during standing" means that the dialogue is not continuing. Show that. The urgency is the urgency of the switching request according to the dialogue purpose and dialogue continuation information when the associated switching condition is satisfied.

In the example shown in FIG. 6, the switching condition that "a person's movement or utterance is not detected for 10 seconds or more" is associated with the dialogue continuation information "during dialogue", the dialogue purpose "greeting", and the "low" urgency. ing. That is, the dialogue is continued when the monitoring display device 2 determines that the switching condition that "the movement and utterance of the person are not detected for 10 seconds or more" is satisfied, and the dialogue purpose of the dialogue is "greeting". If so, a switching request with a "low" urgency is sent.

Further, the monitoring display device 2 does not have to transmit the switching request even when it is determined that the switching condition is satisfied according to the dialogue continuation information. For example, even if the monitoring display device 2 fails to recognize the utterance of the person detected, if the monitoring display device 2 does not have a dialogue with the person, there is little need to switch to the second response. Therefore, in the example shown in FIG. 6, the switching condition that "the utterance of the person cannot be recognized or the utterance cannot be dealt with" is not associated with the dialogue continuation information of "while standing". That is, even if the switching condition that "the person's utterance cannot be recognized or the person cannot respond to the utterance" is satisfied when the person is not interacting with the person, the monitoring display device 2 does not have to send the switching request. Good. By doing so, it is possible for the operator to respond more appropriately according to the situation of the person in the monitoring space.

The various data in the switching condition table 262 are appropriately set in advance by the security service provider and stored in the first storage unit 26.

Not limited to the example of FIG. 6, for example, when the switching condition is satisfied, the urgency of "high" is set to "high" if the dialogue continuation information is "during dialogue" regardless of the content of the switching condition or the purpose of dialogue. If the dialogue is ongoing during "stand-by", the urgency may be "low". In addition, regardless of the content of the switching conditions and the purpose of the dialogue, the urgency of "greeting" is "low" for the purpose of dialogue, and the urgency of "medium" for the purpose of dialogue of "guidance" is "questioning safety". For the purpose of dialogue, the degree of urgency may be "high". In addition, the degree of urgency in consideration of the content of the switching condition and the dialogue continuation information may be associated. In this case, for example, "during dialogue" in "the person's utterance cannot be recognized or responded to the utterance" is associated with the urgency of "medium", and in "the person repeats the same utterance three or more times". "During dialogue" is associated with a "medium" urgency. In addition, the degree of urgency in consideration of the content of the switching condition and the purpose of dialogue may be associated. For example, when the switching condition of "a person's emotion is" anger "" is satisfied, "greeting" is associated with "low" urgency, and "guidance" is associated with "high" urgency. , And in the case of "safety question", associate the urgency of "medium".

FIG. 7 is a sequence diagram showing an example of the flow of monitoring processing by the security system 1, and FIG. 8 is a flow diagram showing an example of the flow of response processing by the security system 1. In these processes, the first control unit 27 and the second control unit 37 configure each of the monitoring display device 2 and the management device 3 based on the programs stored in the first storage unit 26 and the second storage unit 36 in advance. It is realized by collaborating with the elements. It should be noted that these processes are periodically and asynchronously executed in parallel with each other.

Further, while these processes are being executed, the first display control unit 271 of the monitoring display device 2 continuously displays the guard character on the first display unit 21. In this case, the first display control unit 271 may control the display of the guard character so that the guard character performs a predetermined monitoring operation. The predetermined monitoring operation is, for example, an operation of moving the line of sight or the direction of the face or body to the left or right to monitor the surroundings, or a line of sight to track the person for a predetermined time when a person is detected in the monitoring space. It is an action to move. By doing so, the first display control unit 271 gives a sense of security to the user by the natural movement of the guard character, and promotes the user's motivation to receive the response by the guard character.

First, the monitoring process will be described with reference to FIG. First, the transmission unit 276 of the monitoring display device 2 causes the first imaging unit 22 to image the monitoring space and acquires the captured image (S101). Subsequently, the identification unit 274 identifies the dialogue status between the guard character and the person detected in the monitoring space, and stores it in the first storage unit 26 (S102). The dialogue status includes the latest dialogue continuation information and the purpose of the dialogue. The method of specifying the dialogue situation by the specific unit 274 will be described in detail in the description of the response process. Subsequently, the transmission unit 276 associates the captured image with the dialogue status and transmits the captured image to the management device 3 (S103).

Subsequently, the second display control unit 371 of the management device 3 displays the captured image received from each monitoring display device 2 and the dialogue status in association with each other (S104). For example, the second display control unit 371 displays the monitoring screen 900 including the received captured image and the dialogue status on the second display unit 32.

FIG. 9 is a diagram showing an example of the monitoring screen 900 displayed on the second display unit 32. The monitoring screen 900 includes captured images 910-940 transmitted from each of the monitoring display devices 2a-2d. On the captured image 910, the name 911 of the corresponding monitoring display device 2a and the dialogue continuation information 912 are superimposed and displayed. Further, when the dialogue continues, the dialogue purpose 913 of the monitoring display device 2a may be superimposed and displayed on the captured image 910. The same applies to the captured images 920-940. Instead of displaying the name of the monitoring display device 2, information indicating the installation location (for example, an entrance or the like) may be displayed. Either one of the dialogue continuation information 912 and the dialogue purpose 913 may be displayed.

Subsequently, the response process will be described with reference to FIG. First, guard guarding is performed by controlling the display so that the line of sight of the guard character moves within a predetermined range. In parallel with this control, the detection unit 272 of the monitoring display device 2 detects a person around the monitoring device including the monitoring space (S201). For example, the detection unit 272 detects a person in the monitoring space from the captured image. In this case, the detection unit 272 applies frame subtraction technology or background subtraction technology to a plurality of sequentially captured images, and extracts a change region excluding the background from the captured images. The detection unit 272 extracts a person area by performing a matching process between the size and shape of the extracted change area and the pattern stored in advance. Subsequently, the detection unit 272 refers to the map information stored in the first storage unit 26 in advance, and identifies the person position based on the position corresponding to the foot of the extracted person area.

The detection unit 272 may detect a person from the distance measurement data acquired by the sensor 23. In this case, the detection unit 272 extracts a change region indicating the direction in which the distance has changed with respect to the distance measurement data measured when there is no person in the monitoring space in the acquired distance measurement data. Subsequently, the detection unit 272 extracts the person area by performing a matching process between the size and shape of the extracted change area and the pattern stored in advance. Subsequently, the detection unit 272 identifies the person position based on the position of the extracted person area.

Subsequently, the detection unit 272 determines whether or not a person has been detected in the monitoring space in the detection process of S201 (S202). If it is determined that a person has been detected, the response process proceeds to S203. If no person is detected, the response process returns to S201. The detection unit 272 may determine that the person has been detected when the person is continuously detected in the monitoring space for a predetermined time (for example, 5 seconds) or longer.

Subsequently, the first response unit 273 performs the first response by having the guard character interact with the detected person (S203). The first response unit 273 controls the display of the guard character so that the guard character performs a predetermined response operation in the first response. The predetermined response action is, for example, an action of changing the direction of the body to the direction of the detected person, an action of directing the line of sight to the face of the detected person, and an action of giving a bow to the detected person. The predetermined response action may include different actions depending on the attributes of the detected person, such as squatting to adjust the height of the line of sight when the detected person is a child. By doing so, the first display control unit 271 can give a sense of security to the user who receives the response by the natural movement of the guard character.

First, the first response unit 273 identifies the detected movement and utterance of the person. For example, the first response unit 273 extracts joint points such as shoulders and elbows from the person area extracted from the captured image, and specifies the positional relationship of the joint points. Then, the first response unit 273 estimates the posture of the person by performing matching processing between the specified positional relationship and the pattern preset for each posture. Then, the first response unit 273 identifies the movement of the person based on the estimated time change of the posture. For example, if the postures of a person are estimated to be "standing", "crouching", and "falling down" in this order, the person moves from the "standing" posture to "falling down". it is conceivable that. Therefore, in this case, the first response unit 273 specifies the motion of "falling down" as the motion of the person. Further, as the movement of the person, it may be determined whether the person is moving from the temporal change of the position of the person. Further, the first response unit 273 specifies the content included in the utterance of a person by applying a known voice recognition technique and a natural language processing technique to the voice data acquired from the first voice input / output unit 24. Subsequently, the first response unit 273 refers to the dialogue purpose table 261 and determines whether or not the action and utterance of the specified person satisfy any of the dialogue conditions. If it is determined that any of the dialogue conditions is satisfied, the first response unit 273 identifies the dialogue purpose corresponding to the dialogue condition. If none of the dialogue conditions are met, the dialogue will not be held and the guard guard will be continued.

Subsequently, the first response unit 273 refers to the dialogue purpose table 261 and acquires the utterance content associated with the specified dialogue purpose. Subsequently, the first response unit 273 outputs the acquired utterance content as voice via the first voice input / output unit 24.

Subsequently, the identification unit 274 identifies the dialogue status between the guard character and the detected person (S204). The specific unit 274 specifies the purpose of the dialogue specified by the first response unit 273 in S203 as the dialogue situation. In addition, the specific unit 274 determines whether or not the dialogue between the guard character and the detected person continues as a dialogue situation. For example, when the dialogue condition that "a person approaches and 5 seconds or more have passed" is satisfied and the guard character speaks, there is only one-sided speech from the guard character, and the dialogue has not started. .. Therefore, in this case, the specific unit 274 determines that the dialogue is not continuing, and stores "while standing" in the first storage unit 26 as the current dialogue continuation information. On the other hand, when the dialogue condition of "questioning the place from the person" is satisfied and the guard character speaks, the dialogue that the guard character responds to the question from the person has started. Therefore, in this case, the specific unit 274 determines that the dialogue is continuing, and stores "during dialogue" in the first storage unit 26 as the current dialogue continuation information. The identification unit 274 specifies the current dialogue continuation information stored in the first storage unit 26 as the dialogue status. The specific unit 274 stores the specified dialogue status in the first storage unit 26.

Subsequently, the determination unit 275 of the monitoring display device determines whether or not the switching condition is satisfied (S205). The determination unit 275 refers to the switching condition table 262 and determines whether or not any of the plurality of switching conditions is satisfied.

First, the determination unit 275 identifies the movement and utterance of the person in the same manner as the first response unit 273. Then, the determination unit 275 determines that the switching condition that "the movement and utterance of the person is not detected for 10 seconds or more" is satisfied when the person does not move for 10 seconds or more and does not speak for 10 seconds or more. .. Further, the determination unit 275 determines that when the acquired utterance of the person includes a specific word such as "painful", the switching condition that "the utterance of the person includes a specific word" is satisfied. Further, the determination unit 275 determines that the switching condition that "the emotion of the person is" anger "" is satisfied when the sound pressure of the voice input related to the utterance of the person is equal to or higher than a predetermined value. Further, the determination unit 275 determines that the switching condition that "the person repeats the same utterance three times or more" is satisfied when the acquired utterances of the person continuously include the same content. Further, the determination unit 275 determines that when the utterances of the guard character continuously include the same content, the switching condition that "the guard character repeats the same utterance three times or more" is satisfied.

The determination unit 275 may determine whether or not the switching condition regarding the emotion of the person is satisfied based on the facial expression of the person. In this case, the determination unit 275 extracts a feature amount such as the position or size of each part such as eyes, ears, and nose from the face area located at the upper part of the person area extracted from the captured image, and preliminarily extracts each facial expression. The facial expression of a person is estimated by performing matching processing with the set feature amount pattern. Subsequently, the emotion of the person is specified based on the facial expression, and it is determined whether or not the switching condition for the specified emotion is satisfied.

When it is determined that any of the switching conditions is satisfied (S205-Y), the transmission unit 276 transmits a request for switching from the first response to the second response by a call with the operator to the management device 3 ( S206). The transmission unit 276 refers to the switching condition table 262, and determines the urgency of the switching request based on the switching condition determined to be satisfied and the dialogue status when the switching condition is satisfied. Subsequently, the transmission unit 276 transmits a switching request in which the determined urgency is set as a parameter to the management device 3.

If it is determined that the switching condition is not satisfied (S205-N), the response process proceeds to S210.

The second display control unit 371 of the management device 3 displays the captured image received from the monitoring display device 2 that has transmitted the switching request in different modes according to the urgency of the switching request (S207). The second display control unit 371 receives the switching request, identifies the monitoring display device 2 that has transmitted the received switching request, and acquires the urgency set in the switching request. The second display control unit 371 displays the captured image transmitted from the specified monitoring display device 2 in a mode according to the degree of urgency acquired.

FIG. 10 is a diagram showing an example of the monitoring screen 1000 displayed on the second display unit 32 in S207. The monitoring screen 1000 shown in FIG. 10 is a screen displayed in place of the monitoring screen 900 when a switching request is transmitted from the monitoring display device 2d. The description of the same configuration as in FIG. 9 will be omitted. The monitoring screen 1000 includes a highlighting object 1001, a second response start screen 1010, and a dialogue history 1020.

The highlighting object 1001 is an object for displaying the captured image of the monitoring display device 2d that has transmitted the switching request in a manner different from the captured image of the other monitoring display device 2a-2c. Further, the display mode of the highlighting object 1001 is changed according to the urgency set in the switching request. The second response start screen 1010 is a screen for causing the operator to start the second response, and includes the urgency information 1011 and the "respond" button 1012 and the "not respond" button 1013.

The display mode of the captured image may be changed by a method different from that of the highlighting object 1001. For example, the captured image of the monitoring display device 2d that has transmitted the switching request may be enlarged and displayed. Further, the captured image of the monitoring display device 2d that has transmitted the switching request may be displayed in the center of the monitoring screen 1000. Further, the display mode may be changed according to the urgency of the switching request. For example, the higher the degree of urgency, the larger the captured image of the monitoring display device 2d that transmitted the switching request is displayed.

Returning to FIG. 8, the second response unit 372 determines whether or not the condition for starting the second response is satisfied (S208). When the second response unit 372 accepts the operation of the operator who selects the "response" button 1012, the second response unit 372 determines that the condition for starting the second response is satisfied. Further, when the second response unit 372 accepts the operation of the operator who selects the "not respond" button 1013, the second response unit 372 determines that the condition for starting the second response is not satisfied. If it is determined that the condition for starting the second response is not satisfied (S208-N), the response process proceeds to S210.

In addition, the second response unit 372 may perform the switching process to the second response according to the urgency of the received switching request. For example, the second response unit 372 determines that the condition for automatically starting the second response is satisfied when the urgency of the received switching request is equal to or higher than a predetermined value. For example, the second response unit 372 automatically starts the second response when the urgency is "high", and presses the "respond" button 1012 when the urgency is "medium" or "low". The second response is started when the operation of the selected operator is accepted. By doing so, the operator can respond quickly in an emergency.

When it is determined that the condition for starting the second response is satisfied (S208-Y), the second response unit 372 starts the second response (S209). The second response unit 372 transmits the operator's voice and image acquired from the second voice input / output unit 34 and the second imaging unit to the monitoring display device 2 via the second communication unit 31, thereby monitoring and displaying the operator. The second response is performed by talking with the person detected by the device 2. In the second response, the second response unit 372 outputs the operator's voice from the first voice input / output unit 24 of the monitoring display device 2, and causes the first display unit 21 to display the operator's image.

FIG. 11 is a diagram showing an example of the response screen 1100 displayed on the first display unit 21 after the second response is started. The response screen 1100 includes an operator screen 1110 that displays an image of the operator in addition to the monitoring space A reflected on the mirror panel 211.

Returning to FIG. 8, the first response unit 273 of the monitoring display device 2 determines whether or not the dialogue end condition is satisfied (S210). The conditions for ending the dialogue include, for example, that a person is no longer detected in the surveillance space, and that a voice containing words that are likely to be emitted at the end of the dialogue (for example, "Thank you", "Goodbye", etc.) is detected. Is. When the second response is performed, the first response unit 273 may automatically determine that the dialogue end condition is satisfied.

When it is determined that the dialogue end condition is satisfied (S210-Y), the first response unit 273 updates the dialogue continuation information to "while standing", ends the dialogue, and ends a series of processes. If it is determined that the dialogue end condition is not satisfied (S210-N), the response process returns to S203.

As described above, in the security system 1 according to the present invention, the second display control unit 371 of the management device 3 displays the captured image received from each monitoring display device 2 and the dialogue status in association with each other. By doing so, the security system 1 enables the security guard, who is an operator, to easily visually recognize the response status in each monitoring space.

Further, in the security system 1 according to the present invention, when the second display control unit 371 receives the switching request from the monitoring display device 2, the display mode of the image received from the monitoring display device 2 that has transmitted the switching request is switched. Change according to the urgency of the request. By doing so, the operator can easily visually recognize the urgency of the response when the response by the operator is started.

Further, in the security system 1 according to the present invention, an example of displaying the dialogue status together with the captured image has been described in FIG. 9, but the management device 3 normally displays only the captured image received from each monitoring display device 2. May be good. In this case, the dialogue status received from each monitoring display device 2 is stored in the second storage unit 36, and the dialogue status corresponding to the monitoring display device that transmitted the switching request when the switching request is received is stored in the second storage unit. It may be read from 36 and displayed together with the captured image. As a result, the operator can check the status of the monitoring space at an appropriate timing. Note that the transmission unit 276 may normally transmit only the captured image, and may also transmit the dialogue status when transmitting the switching request.

Further, in the security system 1 according to the present invention, the contents of dialogues spoken by the guard character and the person detected in the monitoring space may be stored in time series as a dialogue history. In this case, the transmission unit 276 associates the captured image with the dialogue history and transmits the dialogue history to the management device 3, and when the second display control unit 371 receives the dialogue history from the monitoring display device 2, the monitoring display device 2 sends the dialogue history. The dialogue history is associated with the received captured image and displayed. By doing so, the security system 1 enables the operator to grasp the status of the past dialogue when the response by the operator is started, and enables smooth response. The dialogue history may be displayed at all times, or may be displayed by an operation by the operator. Further, when the switching request is received, the dialogue history corresponding to the monitoring display device to which the switching request is transmitted may be displayed. As a result, the operator can check the status of the monitoring space at an appropriate timing. The transmission unit 276 may normally transmit only the captured image, and may also transmit the dialogue history when transmitting the switching request.

Further, in the security system 1 according to the present invention, the operator's voice is converted into the character's voice while the image of the guard character is displayed without displaying the operator's image on the first display unit 21, and the monitoring display device 2 It may be output from the first audio input / output unit 24 of. In this case, the voice is converted by the first control unit 27 or the second control unit 37. As a result, it is possible to give the impression that the character in front of the user is responding even when the operator is responding. Further, depending on the selection operation by the operator, the switching condition, and the degree of urgency, it may be determined whether the second response is performed by displaying the image of the operator or by displaying the character. For example, when "the person's emotions are" angry "" or "high" urgency, the operator's image is displayed, and when "the person's movement or utterance is not detected for 10 seconds or more" or "low" In the case of urgency, the second response is performed with the character displayed without displaying the operator's image. As a result, it is possible to take an appropriate response depending on the case where it is preferable to display the image of the corresponding operator or the case where it is preferable to give the impression that the character is responding.

Further, in the security system 1 according to the present invention, the second display control unit 371 may change the display mode of the image received from the monitoring display device 2 according to the dialogue status received from the monitoring display device 2. Good. For example, by enlarging the image of the monitoring display device "during dialogue" or displaying it in the center of the monitoring screen 1000, the operator can easily visually recognize the response status of the monitoring display device 2. Become.

It will be appreciated by those skilled in the art that various changes, substitutions and modifications can be made to this without departing from the spirit and scope of the invention. For example, the above-mentioned processing of each part may be executed in an appropriately different order within the scope of the present invention. In addition, the above-described embodiments and modifications may be carried out in appropriate combinations within the scope of the present invention.

1 Security system 2 Monitoring display device 271 1st display control unit 272 Detection unit 273 1st response unit 274 Specific unit 275 Judgment unit 276 Transmission unit 3 Management device 371 2nd display control unit 372 2nd response unit

Claims (7)

A security system having a plurality of monitoring display devices and a management device communicatively connected to the plurality of monitoring display devices.
Each of the plurality of monitoring display devices
The first display unit that displays an image of a character that imitates a standing guard, and
A voice input / output unit that outputs the voice of the character and inputs the surrounding voice,
An imaging unit that captures the surveillance space and
A detection unit that detects a person in the monitoring space,
A first response unit that performs a first response by causing the character to interact with the detected person via the voice input / output unit.
A specific part that identifies the dialogue status of the dialogue,
A transmission unit that associates an image captured by the imaging unit with the dialogue status and transmits the image to the management device is provided.
The management device includes a second display unit that displays an image received from each monitoring display device in association with the dialogue status.
A security system that features that. The security system according to claim 1, wherein the specific unit specifies whether or not the character is in a dialogue with the detected person as the dialogue situation. Each of the plurality of monitoring display devices further includes a determination unit for determining whether or not a predetermined switching condition relating to the detected movement and utterance of the person or the dialogue is satisfied.
When it is determined that the switching condition is satisfied, the transmitting unit transmits a request for switching from the first response to the second response by a call with an operator who operates the management device to the management device.
The management device is
A call unit that enables the call with the detected person via the monitor display device, and a call unit.
A second response unit that performs switching processing to the second response based on the received switching request is provided.
When the switching request is received, the second display unit displays the dialogue status in association with the image received from the monitoring display device that transmitted the switching request.
The security system according to claim 1 or 2. The transmitting unit transmits the dialogue history showing the dialogue content between the character and the detected person by the first response unit to the management device in association with the image.
The second display unit displays the dialogue history corresponding to the monitoring display device together with the image received from the monitoring display device.
The security system according to any one of claims 1 to 3. The transmitter sets the urgency of the switching request,
The second display unit changes the display mode of the image received from the monitoring display device that transmitted the switching request according to the urgency of the switching request.
The security system according to claim 3. The second display unit changes the display mode of the image received from the monitoring display device according to the dialogue status received from the monitoring display device.
The security system according to any one of claims 1 to 5. The image in the monitoring space captured by each monitoring display device received from multiple monitoring display devices, and the dialogue between the character imitating a standing guard and the person in the monitoring space displayed by each monitoring display device. It is a monitoring display device that is communicably connected to a management device that has a second display unit that displays the status in association with each other.
The first display unit that displays an image of a character that imitates a standing guard, and
A voice input / output unit that outputs the voice of the character and inputs the surrounding voice,
An imaging unit that captures the surveillance space and
A detection unit that detects a person in the monitoring space,
A first response unit that performs a first response by causing the character to interact with the detected person via the voice input / output unit.
A specific part that identifies the dialogue status of the dialogue,
A transmission unit that associates an image captured by the imaging unit with the dialogue status and transmits the image to the management device.
A monitoring display device characterized by comprising.

Images