JP7340063B2 - Security system and surveillance display - Google Patents

Description

The present invention relates to security systems and surveillance display devices.

2. Description of the Related Art Conventionally, security guards are assigned to facilities such as airports, stations, and other transportation facilities, commercial facilities, entertainment facilities, corporate offices, and the like, and perform guard patrol security and the like. Security guards at such facilities not only deter crimes through the intimidation effect brought about by surveillance, but also improve services at facilities by providing guidance and other support to facility users. have.

On the other hand, due to the shortage of security guards, a plurality of surveillance cameras are installed in facilities instead of arranging security guards. In this case, guards stationed in surveillance rooms inside and outside the facility monitor the surveillance space in the facility by checking the images of the plurality of surveillance cameras. Patent Literature 1 describes a surveillance system that improves surveillance efficiency by determining the importance of a surveillance camera based on its installation location and displaying video data from the surveillance camera according to the importance.

JP 2008-278368 A

By the way, as described above, security guards also have the task of providing guidance to users of the facility, but monitoring cameras cannot replace such services. Therefore, the introduction of a monitoring display device having a function of automatically responding to users is under consideration. When such a monitoring display device is introduced, when it becomes difficult to respond using the monitoring display device, the operator will respond to the user by remote call or the like or by going directly to the monitoring space. Therefore, the monitoring display device is required to allow the operator to easily grasp the situation of each monitored space so that the operator can respond smoothly.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a security system and a monitoring display device that enable an operator to easily grasp the situation of each monitored space.

A security system according to the present invention is a security system having a plurality of monitoring display devices and a management device communicably connected to the plurality of monitoring display devices, and each of the plurality of monitoring display devices stands guard. A first display unit for displaying an image of a character imitating a security guard, an audio input/output unit for outputting the voice of the character and inputting surrounding voices, an imaging unit for capturing an image of the monitored space, and an image of the monitored space. a detection unit that detects a person; a first response unit that performs a first response by causing a character to interact with the detected person via a voice input/output unit; a specification unit that specifies a dialogue situation of the dialogue; a transmission unit that associates the image captured by the unit with the dialogue situation and transmits the image to the management device, and the management device displays the image received from each monitoring display device and the dialogue situation in association with each other. It is characterized by comprising two display units.

Further, in the security system according to the present invention, it is preferable that the specifying unit specifies whether or not the character is having a dialogue with the detected person as the dialogue situation.

Further, in the security system according to the present invention, each of the plurality of monitoring display devices further includes a determination unit that determines whether or not a predetermined switching condition regarding the motion and utterance or dialogue of the detected person is satisfied. When it is determined that the switching condition is satisfied, the unit transmits to the management device a request for switching from the first reception to the second reception by talking with the operator who operates the management device, and the management device displays the monitoring display device. and a second response unit that performs switching processing to the second response based on the received switching request. is received, it is preferable that the image received from the monitoring display device that transmitted the switching request is displayed in association with the dialogue situation.

Further, in the security system according to the present invention, the transmission unit transmits a dialogue history indicating the content of the dialogue between the character and the detected person by the first reception unit in association with the image to the management device, and the second display unit Preferably, the corresponding interaction history is displayed on the monitor display device along with the images received from the monitor display device.

Further, in the security system according to the present invention, the transmission unit sets the urgency level of the switching request, and the second display unit displays the display mode of the image received from the monitoring display device that transmitted the switching request according to the urgency of the switching request. It is preferable to change according to the degree.

A monitoring display device according to the present invention includes an image in a monitoring space captured by each monitoring display device received from a plurality of monitoring display devices, and a character modeled after a security guard standing guard displayed by each monitoring display device. a monitoring display device communicably connected to a management device having a second display unit for displaying the status of a dialogue with a person in a monitoring space in association with each other, the monitoring display device being communicatively connected to a character imitating a security guard standing guard. a first display unit for displaying an image, an audio input/output unit for outputting character audio and inputting ambient audio, an imaging unit for imaging a monitored space, a detection unit for detecting a person in the monitored space, A first response unit that performs a first response by causing a character to interact with the detected person via a voice input/output unit, a specification unit that specifies the dialogue status of the dialogue, and an image captured by an imaging unit. and a transmission unit that associates the conversation status with the conversation status and transmits the information to the management device.

The security system and monitoring display device according to the present invention enable the operator to easily grasp the situation of each monitored space.

1 is a diagram showing an example of a schematic configuration of a security system 1; FIG. 2 is a diagram showing an example of a schematic configuration of a monitoring display device 2; FIG. 3 is a schematic diagram for explaining an example of the appearance of the monitoring display device 2; FIG. 3 is a diagram showing an example of a schematic configuration of a management device 3; FIG. FIG. 11 is a diagram showing an example of the data structure of a dialogue purpose table 261; FIG. FIG. 11 is a diagram showing an example of the data structure of a switching condition table 262; FIG. 4 is a sequence diagram showing an example of the flow of monitoring processing by the security system 1; FIG. 4 is a flow diagram showing an example of the flow of response processing by the security system 1; FIG. 9 is a diagram showing an example of a monitoring screen 900; FIG. 10 is a diagram showing an example of a monitoring screen 1000; FIG. It is a figure which shows an example of the response screen 1100. FIG.

Various embodiments of the present invention will now be described with reference to the drawings. However, it should be noted that the technical scope of the present invention is not limited to those embodiments, but extends to the invention described in the claims and equivalents thereof.

FIG. 1 is a diagram showing an example of a schematic configuration of a security system 1 according to the present invention. The security system 1 is used, for example, to provide security services including a monitoring function and a reception function at a facility to be monitored.

The security system 1 has a plurality of monitoring display devices 2a to 2d (hereinafter referred to as monitoring display devices 2 when there is no particular need to distinguish between them) and a management device 3. FIG. The monitoring display device 2 and the management device 3 are communicably connected to each other via a network 4 such as the Internet or an intranet.

The monitoring display device 2 is installed in each different monitoring space in the facility to be monitored, monitors each monitoring space, and provides guidance to the users of the facility.

The management device 3 is an information processing device such as a workstation or a PC. The management device 3 may be a multifunctional mobile phone, a tablet terminal, or the like capable of performing predetermined information processing. The management device 3 is installed, for example, in a monitoring room or the like in the facility, and operated by an operator stationed in the monitoring room. The management device 3 may be installed outside the facility.

The monitoring display device 2 displays a character imitating a security guard standing guard (hereafter referred to as a "security guard character"). The monitoring display device 2 detects a person in the monitored space, such as a facility user. The monitoring display device 2 recognizes the motion and speech of the detected person, and responds by causing the security guard character to interact with the detected person (hereinafter referred to as "first response"). In addition, the monitoring display device 2 has a video call function with the management device 3, and an operator who operates the management device 3 responds to the detected person by transmitting a switching request to the management device 3 (hereinafter referred to as " (referred to as "second response").

FIG. 2 is a diagram showing an example of a schematic configuration of the monitoring display device 2. As shown in FIG. The monitoring display device 2 includes a first display unit 21, a first imaging unit 22, a sensor 23, a first audio input/output unit 24, a first communication unit 25, a first storage unit 26, and a first control unit. a portion 27;

The first display unit 21 is a mirror-type display, and displays information such as images and text according to data supplied from the first control unit 27 . In particular, the first display unit 21 displays the security guard character according to instructions from the first control unit 27 . Note that the first display unit 21 may be a non-mirror type display such as a liquid crystal display or an organic EL display.

The first imaging unit 22 includes an imaging optical system, an imaging device, an image processing unit, and the like. The imaging optical system is, for example, an optical lens, and forms an image of the light flux from the subject on the imaging surface of the imaging element. The imaging device is, for example, a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor), and outputs an image signal of a subject image formed on an imaging surface. The image processing section generates image data in a predetermined file format from the image signal generated by the imaging device, and supplies the image data to the first control section 27 .

The sensor 23 is a device that measures the distance to an object around the monitoring display device 2, and is, for example, a TOF (Time of Flight) type laser ranging sensor. The sensor 23 irradiates pulse-modulated laser light in each direction within a predetermined scanning range, and calculates the distance to the reflection point in each direction based on the time until the reflected light is detected. Subsequently, the sensor 23 associates the direction in which the laser light is irradiated with the distance to the reflection point in each direction, and supplies it to the first control unit 27 as distance measurement data.

The first audio input/output unit 24 is, for example, a microphone and a speaker. The first audio input/output unit 24 converts sounds around the monitoring display device 2 into electric signals and supplies the electric signals to the first control unit 27, and converts the electric signals supplied from the first control unit 27 into sounds. output.

The first communication unit 25 includes a communication interface circuit that enables the monitoring display device 2 to communicate with the management device 3 via the network 4 . The communication interface circuit included in the first communication unit 25 is a communication interface circuit for LTE (Long Term Evolution), a wireless LAN, or a wired LAN. The first communication unit 25 transmits data supplied from the first control unit 27 to the management device 3 , receives data transmitted from the management device 3 , and supplies the data to the first control unit 27 .

The first storage unit 26 is a device for storing programs or data, and includes, for example, a semiconductor memory device. The first storage unit 26 stores an operating system program, a driver program, an application program, data, etc. used for processing by the first control unit 27 . The program is installed in the first storage unit 26 from a computer-readable portable storage medium such as CD-ROM, DVD-ROM, etc. using a known setup program or the like.

The first storage unit 26 also stores a dialogue purpose table 261 and a switching condition table 262 as data.

The first control unit 27 includes one or more processors and their peripheral circuits. The first control unit 27 is, for example, a CPU (Central Processing Unit), and controls the operation of the monitoring display device 2 in a centralized manner. The first control unit 27 may be a DSP (Digital Signal Processor), LSI (Large-Scaled IC), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), or the like. The first control unit 27 controls various processes of the monitoring display device 2 to be executed in an appropriate procedure based on the program stored in the first storage unit 26 or the voice input from the first voice input/output unit 24. , controls the operations of the first display unit 21 and the first communication unit 25 . The first control unit 27 executes processing based on programs stored in the first storage unit 26 . Also, the first control unit 27 can execute a plurality of programs in parallel.

The first control unit 27 includes a first display control unit 271, a detection unit 272, a first response unit 273, a specification unit 274, a determination unit 275, and a transmission unit 276 as functional blocks. The first display control unit 271 controls the movement of the security guard character to be displayed according to the recognized situation of the monitored space. In addition, when the first response unit 273 detects a person stopping near the monitoring display device 10 (for example, within 1 m) or detects an utterance from the person, the first response unit 273 controls voice output so as to interact with the person. do. The first display control section 271 , the detection section 272 , the first response section 273 , the identification section 274 , the determination section 275 and the transmission section 276 are functional modules realized by the program executed by the first control section 27 . Further, each of these units may be implemented in the monitoring display device 2 as firmware.

FIG. 3 is a schematic diagram for explaining the appearance of the monitoring display device 2. As shown in FIG. As shown in FIG. 3 , the first display section 21 of the monitoring display device 2 is supported by a base 28 . The monitoring display device 2 is arranged so that the first display section 21 faces the monitoring space A. As shown in FIG.

The first display section 21 is formed, for example, by attaching a mirror panel 211 to the display surface of the monitor. The mirror panel 211 transmits light (image) from the inside (monitor side) while reflecting light from the outside (monitoring space side). As a result, the first display unit 21 displays the image of the guard character C output from the first control unit 27 while reflecting and displaying the images of the persons H1 and H2 who come and go in the monitored space A.

A sensor 23 is arranged on the outer frame 212 of the mirror panel 211 so as to be able to irradiate a near-infrared pulse laser toward the monitored space A. As shown in FIG. Similarly, in the outer frame 212, the first imaging unit 22 is arranged so as to be able to image the monitored space A, and the first audio input/output unit 24 is capable of collecting the sound of the monitored space A and directing it to the monitored space A. Arranged so that sound can be output. The arrangement of the sensor 23 , the first imaging section 22 , and the first audio input/output section 24 is not limited to the outer frame 212 , and may be arranged over the mirror panel 211 .

FIG. 4 is a diagram showing an example of a schematic configuration of the management device 3. As shown in FIG. The management device 3 includes a second communication unit 31, a second display unit 32, an operation unit 33, a second audio input/output unit 34, a second imaging unit 35, a second storage unit 36, a second control a portion 37;

The configurations of the second communication unit 31, the second audio input/output unit 34, the second imaging unit 35, and the second storage unit 36 are the first communication unit 25, the first audio input/output unit 24, the first imaging unit 22, and the second storage unit 36, respectively. It is similar to the first storage unit 26 .

The second display unit 32 is a device that displays images and the like to the operator, and is, for example, a liquid crystal display or an organic EL display. The second display section 32 displays an image or the like according to data supplied from the second control section 37 .

The operation unit 33 is a device for receiving an operator's operation of the management device 3, and is, for example, a keyboard and a mouse. The operation unit 33 receives input operations and selection operations of characters, numbers, symbols, etc. by the user. When receiving an operation by the user, the operation unit 33 generates a signal corresponding to the operation and supplies the signal to the second control unit 37 .

The second control unit 37 includes one or more processors and their peripheral circuits. The second control unit 37 is, for example, a CPU, and controls the operation of the management device 3 in a centralized manner. Note that the second control unit 37 may be DSP, LSI, ASIC, FPGA, or the like. The second control unit 37 controls the operations of the second communication unit 31 and the second display unit 32 so that various processes of the management device 3 are executed in appropriate procedures based on programs stored in the management device 3. Control. The second control unit 37 executes processing based on programs stored in the management device 3 . Also, the second control unit 37 can execute a plurality of programs in parallel.

The second control section 37 has a second display control section 371 and a second response section 372 as its functional blocks. Each of these units is a functional module implemented by a program executed by the second control unit 37 . Further, each of these units may be implemented in the management device 3 as firmware.

FIG. 5 is a diagram showing an example of the data structure of the dialogue purpose table 261 stored in the first storage unit 26 of the monitoring display device 2. As shown in FIG. The dialogue purpose table 261 stores a plurality of dialogue conditions, dialogue purposes, and utterance contents in association with each other.

The dialogue condition is a condition for the monitoring display device 2 to have a dialogue in the first response, and is a condition regarding a person's motion or speech detected by the monitoring display device 2 . An interaction objective is the objective of an interaction that occurs if the associated interaction condition is met. Dialogue purposes include "greetings", "guidance", and "inquiry about safety". The utterance content is the content uttered by the security guard character (that is, output as voice) when the associated dialogue condition is satisfied, and the content in line with the associated dialogue purpose is stored.

In the example shown in FIG. 5, the dialogue condition "5 seconds or more have passed since the person approached" is associated with the dialogue purpose "greeting" and the utterance content "hello." That is, when the monitoring display device 2 determines that the dialogue condition that "a person has been approaching for 5 seconds or more" is satisfied, a dialogue with a "greeting" as the purpose of the dialogue is started in the first response. Then, the monitoring display device 2 causes the security guard character to say "Hello." Note that the detection unit 272 determines that the person is approaching the specific area when the position of the person in the monitored space is within a predetermined distance from the position of the specific area set in advance.

Similarly, when the monitoring display device 2 determines that the dialogue condition of "inquiry about a place from a person" is satisfied, it starts a dialogue whose purpose is "guidance" in the first response. Then, the monitoring display device 2 acquires the route to the place where the person asked from the first storage unit 26, and tells the security guard character, "Please follow the passage on the right to XX." , to guide the obtained route.

Various data in the dialogue purpose table 261 are appropriately set in advance by the security service provider and stored in the first storage unit 26 .

FIG. 6 is an example of the data structure of the switching condition table 262 stored in the first storage unit 26. As shown in FIG. The switching condition table 262 stores switching conditions, dialogue continuation information, dialogue purposes, and degrees of urgency in association with each other.

The switching condition is a condition for the monitoring display device 2 to transmit a request for switching from the first response to the second response to the management device, and the motion and speech of the person detected by the monitoring display device 2 or the detected person and is a condition regarding the dialogue in the first response of . The dialogue continuation information is information indicating whether or not the dialogue is continuing in the first response, and "under dialogue" indicates that the dialogue is continuing, and "under guard" means that the dialogue is not continuing. indicates that The urgency is the urgency of the switching request according to the dialogue purpose and dialogue continuation information when the associated switching condition is satisfied.

In the example shown in FIG. 6, the switching condition of "a person's motion or speech is not detected for 10 seconds or more" is associated with the dialogue continuation information of "dialogue in progress", the dialogue purpose of "greeting", and the degree of urgency of "low". ing. That is, when the monitoring display device 2 determines that the switching condition that "a person's motion or speech is not detected for 10 seconds or more" is satisfied, the dialogue continues, and the purpose of the dialogue is a "greeting." For example, a switching request with a "low" degree of urgency is transmitted.

Further, the monitoring display device 2 may not transmit a switching request even when it is determined that the switching condition is satisfied according to the dialogue continuation information. For example, even if the monitoring display device 2 fails to recognize the utterance of a detected person, if the monitoring display device 2 is not interacting with that person, there is little need to switch to the second response. Therefore, in the example shown in FIG. 6, the dialogue continuation information "on patrol" is not associated with the switching condition "unable to recognize the person's speech or respond to the speech". In other words, even if the switching condition that "a person's speech cannot be recognized or cannot be responded to" is satisfied when the person is not interacting with the person, the monitoring display device 2 does not transmit a switching request. good. By doing so, it becomes possible for the operator to respond more appropriately according to the situation of the person in the monitored space.

Various data in the switching condition table 262 are appropriately set in advance by the security service provider and stored in the first storage unit 26 .

In addition, not limited to the example of FIG. If the dialogue is in a continuation state of "Standing on Patrol", the urgency level may be set to "Low". In addition, regardless of the content of the switching conditions or the dialogue purpose, the urgency level is set to "low" for the dialogue purpose of "greeting", and "medium" for the dialogue purpose of "guidance". If it is for the purpose of dialogue, the urgency may be set to "high". In addition, the urgency may be associated with the content of the switching condition and the dialogue continuation information. In this case, for example, "during a dialogue" in "cannot recognize or respond to a person's utterance" is associated with a "medium" urgency, and "a person repeats the same utterance three or more times" "Interacting" is associated with a "medium" urgency. Also, the content of the switching condition and the degree of urgency considering the purpose of the dialogue may be associated. For example, when the switching condition "a person's emotion is 'anger'" is satisfied, the urgency level of 'low' is associated with 'greeting' as the dialogue purpose, and 'high' with 'guidance'. , and in the case of "safety inquiry", it is associated with "medium" urgency.

FIG. 7 is a sequence diagram showing an example of the flow of monitoring processing by the security system 1, and FIG. 8 is a flowchart showing an example of the flow of response processing by the security system 1. As shown in FIG. These processes are performed by the first control unit 27 and the second control unit 37 controlling each component of the monitoring display device 2 and the management device 3 based on programs stored in advance in the first storage unit 26 and the second storage unit 36 . It is realized by cooperating with the elements. Note that these processes are periodically executed in parallel and asynchronously with each other.

Further, while these processes are being executed, the first display control section 271 of the monitoring display device 2 continues to display the guard character on the first display section 21 . In this case, the first display control unit 271 may control the display of the security guard character so that the security guard character performs a predetermined monitoring action. The predetermined monitoring action is, for example, an action of moving the line of sight or the orientation of the face or body to the left or right to monitor the surroundings, or moving the line of sight to track the person for a predetermined period of time when a person is detected in the monitored space. It is an action to move. By doing so, the first display control unit 271 gives the user a sense of security through the natural movement of the security guard character, and promotes the user's willingness to receive a response from the security guard character.

First, the monitoring process will be described with reference to FIG. First, the transmission unit 276 of the monitoring display device 2 causes the first imaging unit 22 to capture an image of the monitored space, and obtains a captured image (S101). Subsequently, the specifying unit 274 specifies the dialogue situation between the guard character and the person detected in the monitored space, and stores it in the first storage unit 26 (S102). The dialogue status includes the latest dialogue continuation information and dialogue purpose. A method of specifying the dialogue situation by the specifying unit 274 will be described in detail in the explanation of the handling process. Subsequently, the transmission unit 276 associates the captured image with the dialogue situation and transmits them to the management device 3 (S103).

Subsequently, the second display control unit 371 of the management device 3 displays the captured images received from the respective monitoring display devices 2 in association with the dialogue status (S104). For example, the second display control unit 371 displays the monitoring screen 900 including the received captured image and the conversation situation on the second display unit 32 .

FIG. 9 is a diagram showing an example of a monitoring screen 900 displayed on the second display section 32. As shown in FIG. The monitor screen 900 includes captured images 910-940 transmitted from the monitor display devices 2a-2d, respectively. On the captured image 910, the name 911 of the corresponding monitoring display device 2a and dialogue continuation information 912 are displayed in a superimposed manner. Further, when the dialogue continues, the dialogue purpose 913 of the monitoring display device 2a may be superimposed on the captured image 910 and displayed. The same applies to the captured images 920-940. Instead of displaying the name of the monitoring display device 2, information indicating the installation location (for example, the entrance) may be displayed. Either one of the dialogue continuation information 912 and the dialogue purpose 913 may be displayed.

Next, the handling process will be described with reference to FIG. First, guarding is performed by performing display control so that the line of sight of the guard character is moved within a predetermined range. In parallel with this control, the detection unit 272 of the monitoring display device 2 detects a person around the monitoring device including the monitoring space (S201). For example, the detection unit 272 detects a person in the monitored space from the captured image. In this case, the detection unit 272 applies the frame difference technique or the background difference technique to a plurality of captured images that are sequentially captured, and extracts a changed region from the captured images with the background removed. The detection unit 272 extracts a person area by performing matching processing between the size, shape, etc. of the extracted changed area and a pattern stored in advance. Subsequently, the detection unit 272 refers to the map information stored in the first storage unit 26 in advance, and specifies the person's position based on the position corresponding to the feet of the extracted person area.

Note that the detection unit 272 may detect a person from the distance measurement data acquired by the sensor 23 . In this case, the detection unit 272 extracts, from the acquired distance measurement data, a change area indicating the direction in which the distance changes with respect to the distance measurement data measured when there is no person in the monitored space. Subsequently, the detection unit 272 extracts a person area by performing matching processing between the size, shape, etc. of the extracted changed area and patterns stored in advance. Subsequently, the detection unit 272 identifies the person's position based on the extracted position of the person's region.

Subsequently, the detection unit 272 determines whether or not a person has been detected in the monitored space in the detection processing of S201 (S202). If it is determined that a person has been detected, the processing proceeds to S203. If no person is detected, the response process returns to S201. Note that the detection unit 272 may determine that a person has been detected when a person is continuously detected in the monitored space for a predetermined period of time (for example, 5 seconds).

Subsequently, the first response unit 273 performs the first response by causing the security guard character to interact with the detected person (S203). Note that the first response unit 273 controls the display of the security guard character so that the security guard character performs a predetermined response action in the first response. The predetermined response action is, for example, an action of changing the orientation of the body in the direction of the detected person, an action of directing the line of sight to the detected person's face, or an action of bowing to the detected person. The predetermined response action may include a different action depending on the attribute of the detected person, such as squatting down to match eye level when the detected person is a child. By doing so, the first display control unit 271 can give the user who receives the reception a sense of security through the natural movement of the security guard character.

First, the first response unit 273 identifies the motion and utterance of the detected person. For example, the first response unit 273 extracts joint points such as shoulders and elbows from the human region extracted from the captured image, and specifies the positional relationship of the joint points. Then, the first response unit 273 performs matching processing between the specified positional relationship and a pattern preset for each posture, thereby estimating the posture of the person. Then, the first response unit 273 identifies the motion of the person based on the estimated temporal change of the posture. For example, when the postures of "standing", "crouching", and "falling down" are estimated in this order as the postures of a person, the person made the action of "falling down" from the "standing" posture. it is conceivable that. Therefore, in this case, the first response unit 273 identifies the action of "falling down" as the action of the person. Further, as the action of the person, it may be determined whether the person is moving based on the change in the position of the person over time. Also, the first response unit 273 identifies the content included in the person's utterance by applying known speech recognition technology and natural language processing technology to the voice data acquired from the first voice input/output unit 24 . Subsequently, the first response unit 273 refers to the dialogue purpose table 261 and determines whether or not the motion and speech of the specified person satisfy any dialogue condition. When it is determined that any dialogue condition is satisfied, the first response unit 273 identifies the purpose of dialogue corresponding to that dialogue condition. If none of the dialogue conditions are met, no dialogue will take place, and the guard will continue to stand guard.

Next, the first response unit 273 refers to the dialogue purpose table 261 and acquires the utterance content associated with the specified dialogue purpose. Subsequently, the first response unit 273 causes the first voice input/output unit 24 to output the acquired utterance content as voice.

Subsequently, the identifying unit 274 identifies the dialogue situation between the security guard character and the detected person (S204). The identifying unit 274 identifies, as the dialogue situation, the dialogue purpose identified by the first responding unit 273 in S203. Further, the specifying unit 274 determines whether or not the dialogue between the security guard character and the detected person continues as the dialogue situation. For example, when the dialogue condition that "a person has been approaching for more than 5 seconds" is satisfied and the security guard character speaks, the dialogue is not yet started. . Therefore, in this case, the identifying unit 274 determines that the dialogue is not continuing, and stores "Standing on guard" in the first storage unit 26 as the current dialogue continuation information. On the other hand, when the dialogue condition "question from the person about the location" is satisfied and the security guard character speaks, a dialogue has started in which the security guard character responds to the question from the person. Therefore, in this case, the identification unit 274 determines that the dialogue is continuing, and stores "dialogue in progress" in the first storage unit 26 as the current dialogue continuation information. The identifying unit 274 identifies the current dialogue continuation information stored in the first storage unit 26 as the dialogue situation. The specifying unit 274 stores the specified dialogue situation in the first storage unit 26 .

Subsequently, the determination unit 275 of the monitoring display device determines whether or not the switching condition is satisfied (S205). The determination unit 275 refers to the switching condition table 262 and determines whether or not any one of the plurality of switching conditions is satisfied.

First, the determination unit 275 identifies the motion and utterance of the person in the same manner as the first response unit 273 . Then, when the person does not move for 10 seconds or longer and does not speak for 10 seconds or longer, the determination unit 275 determines that the switching condition that "human motion and speech are not detected for 10 seconds or longer" is satisfied. . In addition, when the acquired speech of the person includes a specific word such as “painful”, the determination unit 275 determines that the switching condition “the speech of the person includes the specific word” is satisfied. Further, the determination unit 275 determines that the switching condition that "the person's emotion is 'anger'" is satisfied when the sound pressure of the voice input related to the person's utterance is equal to or greater than a predetermined value. In addition, when the acquired utterances of the person continuously include the same content, the determination unit 275 determines that the switching condition “the person repeats the same utterance three times or more” is satisfied. Further, when the utterances of the security guard character continuously include the same content, the determination unit 275 determines that the switching condition that "the security guard character repeats the same utterance three times or more" is satisfied.

Note that the determination unit 275 may determine whether or not a switching condition regarding a person's emotion is satisfied based on the person's facial expression. In this case, the determination unit 275 extracts the feature amount such as the position or size of each part such as eyes, ears, nose, etc. from the face region located above the person region extracted from the captured image, A person's facial expression is estimated by performing matching processing with the pattern of the set feature amount. Subsequently, the emotion of the person is identified based on the facial expression, and it is determined whether or not the switching condition regarding the identified emotion is satisfied.

If it is determined that any one of the switching conditions is satisfied (S205-Y), the transmitting unit 276 transmits a request for switching from the first response to the second response by talking with the operator to the management device 3 ( S206). The transmission unit 276 refers to the switching condition table 262 and determines the urgency of the switching request based on the switching condition determined to be satisfied and the dialogue situation when the switching condition is satisfied. Subsequently, the transmitting unit 276 transmits to the management device 3 a switching request in which the determined urgency is set as a parameter.

If it is determined that the switching condition is not satisfied (S205-N), the processing proceeds to S210.

The second display control unit 371 of the management device 3 displays the captured image received from the monitoring display device 2 that transmitted the switching request in different modes according to the degree of urgency of the switching request (S207). The second display control unit 371 receives the switching request, identifies the monitoring display device 2 that transmitted the received switching request, and acquires the degree of urgency set in the switching request. The second display control unit 371 displays the captured image transmitted from the specified monitoring display device 2 in a manner according to the acquired urgency.

FIG. 10 is a diagram showing an example of the monitoring screen 1000 displayed on the second display unit 32 in S207. A monitoring screen 1000 shown in FIG. 10 is a screen displayed instead of the monitoring screen 900 when a switching request is transmitted from the monitoring display device 2d. Note that description of the same configuration as in FIG. 9 will be omitted. A monitoring screen 1000 includes a highlighting object 1001 , a second response start screen 1010 , and a dialogue history 1020 .

The highlighting object 1001 is an object for displaying the captured image of the monitoring display device 2d that has transmitted the switching request in a manner different from the captured images of the other monitoring display devices 2a to 2c. Also, the display mode of the highlighting object 1001 is changed according to the degree of urgency set in the switching request. Second response start screen 1010 is a screen for causing the operator to start the second response, and includes urgency level information 1011 , “respond” button 1012 , and “not respond” button 1013 .

Note that the display mode of the captured image may be changed by a method different from that of the highlighted display object 1001 . For example, the captured image of the monitoring display device 2d that has transmitted the switching request may be enlarged and displayed. Also, the captured image of the monitoring display device 2 d that has transmitted the switching request may be displayed in the center of the monitoring screen 1000 . Furthermore, the display mode may be changed according to the urgency of the switching request. For example, the higher the degree of urgency, the larger the image captured by the monitoring display device 2d that transmitted the switching request is displayed.

Returning to FIG. 8, the second response unit 372 determines whether or not the conditions for starting the second response are satisfied (S208). When the second response unit 372 receives the operator's operation to select the "respond" button 1012, the second response unit 372 determines that the conditions for starting the second response are satisfied. In addition, when the second response unit 372 receives the operator's operation to select the "no response" button 1013, the second response unit 372 determines that the conditions for starting the second response are not satisfied. If it is determined that the condition for starting the second response is not satisfied (S208-N), the response process proceeds to S210.

Note that the second response unit 372 may perform switching processing to the second response according to the degree of urgency of the received switching request. For example, when the urgency level of the received switching request is equal to or greater than a predetermined value, the second response unit 372 determines that the condition for automatically starting the second response is satisfied. For example, the second response unit 372 automatically starts the second response when the urgency is "high", and clicks the "response" button 1012 when the urgency is "medium" or "low". The second response is started when the operation of the operator to select is received. By doing so, it becomes possible for the operator to respond quickly in an emergency.

If it is determined that the condition for starting the second response is satisfied (S208-Y), the second response unit 372 starts the second response (S209). The second response unit 372 transmits the operator's voice and image obtained from the second voice input/output unit 34 and the second imaging unit to the monitoring display device 2 via the second communication unit 31, thereby A second response is performed by talking with the person detected by the device 2 . In the second response, the second response section 372 outputs the operator's voice from the first voice input/output section 24 of the monitoring display device 2 and displays the operator's image on the first display section 21 .

FIG. 11 is a diagram showing an example of a response screen 1100 displayed on the first display section 21 after the second response is started. The response screen 1100 includes an operator screen 1110 that displays an image of the operator in addition to the monitored space A reflected on the mirror panel 211 .

Returning to FIG. 8, the first response unit 273 of the monitoring display device 2 determines whether or not the dialogue end condition is satisfied (S210). The conditions for ending the dialogue are, for example, that no person is detected in the monitored space, that a voice containing words that are likely to be uttered at the end of the dialogue (for example, "thank you", "goodbye", etc.) is detected, etc. is. In addition, when the second response is performed, the first response unit 273 may automatically determine that the dialogue end condition is satisfied.

When it is determined that the dialogue end condition is satisfied (S210-Y), the first response unit 273 updates the dialogue continuation information to "Still on guard", ends the dialogue, and ends the series of processes. If it is determined that the dialogue end condition is not satisfied (S210-N), the processing returns to S203.

As described above, in the security system 1 according to the present invention, the second display control unit 371 of the management device 3 displays the captured images received from the respective monitoring display devices 2 and the dialogue status in association with each other. By doing so, the security system 1 enables the security guard who is the operator to easily visually recognize the response status in each monitored space.

Further, in the security system 1 according to the present invention, when receiving a switching request from the monitoring display device 2, the second display control unit 371 switches the display mode of the image received from the monitoring display device 2 that has transmitted the switching request. Change according to the urgency of the request. By doing so, the operator can easily visually recognize the urgency of the response when the operator starts to respond.

In addition, in the security system 1 according to the present invention, an example in which the conversation status is displayed together with the captured image has been described with reference to FIG. good too. In this case, the dialogue status received from each monitoring display device 2 is stored in the second storage unit 36, and when a switching request is received, the dialogue status corresponding to the monitoring display device that transmitted the switching request is stored in the second storage unit 36. 36 and displayed together with the captured image. This allows the operator to check the status of the monitored space at an appropriate timing. It should be noted that the transmission unit 276 may normally transmit only the captured image, and may also transmit the conversation status when transmitting the switching request.

In addition, in the security system 1 according to the present invention, the content of conversations between the security guard character and the person detected in the monitored space may be stored in chronological order as a conversation history. In this case, the transmission unit 276 associates the dialogue history with the captured image and transmits it to the management device 3 , and when the second display control unit 371 receives the dialogue history from the monitoring display device 2 , The dialogue history is displayed in association with the received captured image. By doing so, the security system 1 enables the operator to grasp the status of the past dialogue when the operator starts to respond, thus enabling smooth response. The dialogue history may be displayed all the time, or may be displayed by an operator's operation. Further, when a switching request is received, the dialogue history corresponding to the monitoring display device that sent the switching request may be displayed. This allows the operator to check the status of the monitored space at an appropriate timing. It should be noted that the transmission unit 276 may normally transmit only the captured image, and transmit the dialogue history together with the switching request.

Further, in the security system 1 according to the present invention, the image of the security guard character is displayed without displaying the image of the operator on the first display unit 21, and the voice of the operator is converted into the voice of the character. may be output from the first audio input/output unit 24 of the In this case, the voice is converted by the first controller 27 or the second controller 37 . As a result, even when the operator is responding, it is possible to give the impression that the character in front of the user is responding. Further, it may be determined whether the second response is performed by displaying the image of the operator or by displaying the character as it is, depending on the selection operation by the operator, the switching condition, and the degree of urgency. For example, if "a person's emotion is 'anger'" or if the urgency level is "high", an image of the operator is displayed. In the case of the degree of urgency, the operator's image is not displayed, and the second response is performed while the character is displayed. As a result, an appropriate response can be made according to the case where it is preferable to display the image of the corresponding operator or the case where it is preferable to give the impression that the character is responding.

Further, in the security system 1 according to the present invention, the second display control unit 371 may change the display mode of the image received from the monitoring display device 2 according to the dialogue status received from the monitoring display device 2. good. For example, by enlarging and displaying the image of the monitoring display device "during dialogue" or displaying it in the center of the monitoring screen 1000, the operator can easily visually recognize the response status of the monitoring display device 2. Become.

It should be understood by those skilled in the art that various changes, substitutions and modifications can be made thereto without departing from the spirit and scope of the present invention. For example, the processing of each unit described above may be performed in a different order as appropriate within the scope of the present invention. Also, the above-described embodiments and modifications may be implemented in appropriate combinations within the scope of the present invention.

1 security system 2 monitoring display device 271 first display control unit 272 detection unit 273 first response unit 274 identification unit 275 determination unit 276 transmission unit 3 management device 371 second display control unit 372 second response unit

Claims (10)

A security system comprising a monitoring display device and a management device communicably connected to the monitoring display device,
The monitoring display device
a first display unit that displays an image of a character;
an imaging unit that captures an image of the monitored space;
a detection unit that detects a person in the monitored space;
a first response unit that controls the character and performs a first response to the detected person;
a specifying unit for specifying a response situation of the first response that the character is performing to the detected person by the first response unit;
a transmitting unit that sequentially transmits the current image captured by the imaging unit and the current response status specified by the specifying unit to the management device;
The management device includes a second display unit that displays the current image sequentially received from the monitoring display device, and that displays the current image and the current response status in association with each other.
A security system characterized by: A security system having a plurality of monitoring display devices and a management device communicably connected to the monitoring display devices,
The monitoring display device
a first display unit that displays an image of a character;
an imaging unit that captures an image of the monitored space;
a detection unit that detects a person in the monitored space;
a first response unit that controls the character and performs a first response to the detected person;
a specifying unit for specifying a response situation of the first response between the character and the detected person by the first response unit;
a transmission unit that transmits the image captured by the imaging unit and the response status to the management device;
The management device includes a second display unit that displays the image received from the monitoring display device and the response situation in association with each other,
The second display unit displays a list of the reception status received from the monitoring display device that transmitted the image together with the image received from each monitoring display device.
A security system characterized by: A security system comprising a monitoring display device and a management device communicably connected to the monitoring display device,
The monitoring display device
a first display unit that displays an image of a character standing guard;
an imaging unit that captures an image of the monitored space;
a detection unit that detects a person in the monitored space;
a first response unit that controls the character and performs a first response to the detected person;
a specifying unit for specifying a response situation of the first response between the character and the detected person by the first response unit;
a transmission unit that transmits the image captured by the imaging unit and the response status to the management device;
The management device includes a second display unit that displays the image received from the monitoring display device and the response situation in association with each other,
The specifying unit specifies, as the response status, whether the character is having a conversation with the detected person or is standing guard without having a conversation.
A security system characterized by: In the second display unit, when the reception status received from the monitoring display device is "dialogue", the visibility of the image received from the monitoring display device is higher than when the reception status is "standing guard". change the display mode of the image so that
Security system according to claim 3 . A security system comprising a monitoring display device and a management device communicably connected to the monitoring display device,
The monitoring display device
a first display unit that displays an image of a character;
an imaging unit that captures an image of the monitored space;
a detection unit that detects a person in the monitored space;
a first response unit that controls the character and performs a first response to the detected person;
a specifying unit for specifying a response situation of the first response between the character and the detected person by the first response unit;
a determination unit that determines whether a predetermined switching condition regarding the detected person's motion and speech or the first response is satisfied;
The image captured by the imaging unit and the response status are transmitted to the management device, and when it is determined that the switching condition is satisfied, the first response is performed through a call with an operator who operates the management device. 2 a transmission unit that transmits a request to switch to response to the management device,
The management device
a call unit that enables the call with the detected person through the monitoring display device;
a second response unit that performs switching processing to the second response based on the received switching request;
a second display unit that, when receiving the switching request, displays the image received from the monitoring display device that transmitted the switching request in association with the response status;
A security system characterized by: The transmission unit sets the urgency of the switching request,
The second display unit changes the display mode of the image received from the monitoring display device that has transmitted the switching request, according to the urgency of the switching request.
Security system according to claim 5 . Having a plurality of the monitoring display devices,
The management device further includes a display control unit that receives images sequentially transmitted from each monitoring display device and acquires the current response status of the monitoring display device that transmitted the image,
The second display unit displays the image received from the monitoring display device in association with the acquired current response status.
A security system according to any one of claims 1-6 . The transmission unit transmits a dialogue history indicating a dialogue content between the character and the detected person by the first reception unit to the management device,
The second display unit displays the correspondence status received from the monitoring display device and the dialogue history received from the monitoring display device in association with each other.
A security system according to any one of claims 1-7. A current image of the monitored space imaged by the monitoring display device and a current response state of the first response performed by the character displayed by the monitoring display device to the person in the monitored space are displayed in association with each other. A monitoring display device communicably connected to a management device comprising two display units,
a first display unit that displays an image of the character;
an imaging unit that captures an image of the monitored space;
a detection unit that detects a person in the monitored space;
a first response unit that controls the character and performs a first response to the detected person;
a specifying unit that specifies a response situation of the first response that the character is performing to the detected person by the first response unit;
a transmitting unit that sequentially transmits the current image captured by the imaging unit and the current response status specified by the specifying unit to the management device;
A monitoring display device comprising: Management comprising a second display unit for displaying an image of the monitored space captured by the monitoring display device in association with the response status of the character guarding the guard displayed by the monitoring display device and the person in the monitored space. A monitoring display device communicatively connected to the device,
a first display unit that displays an image of the character;
an imaging unit that captures an image of the monitored space;
a detection unit that detects a person in the monitored space;
a first response unit that controls the character and performs a first response to the detected person;
a specifying unit for specifying a response situation of the first response between the character and the detected person by the first response unit;
a transmission unit that transmits the image captured by the imaging unit and the response status to the management device;
The specifying unit specifies, as the response status, whether the character is having a conversation with the detected person or is standing guard without having a conversation.
A monitoring display device characterized by:

Images