JP5114924B2 - Warning method by security robot system and security robot system - Google Patents

Description

  The present invention relates to monitoring of a store or the like using a monitoring camera mounted on a robot or a monitoring camera fixed to a wall or the like, and in particular, the customer travels around the store with an autonomous mobile robot mounted with a camera, The present invention relates to a crime prevention system and a crime prevention method for monitoring and warning a suspicious person identified from the above.

There is a crime prevention system in which a surveillance staff monitors a surveillance camera installed at a fixed position such as a wall or ceiling in a store such as a bookstore or a supermarket. In crime prevention by surveillance personnel using fixed-position surveillance cameras, when surveillance personnel find a suspicious person, they go to the site and perform warning and tracking. In this case, there are the following problems.
A. There is a time zone during which monitoring cannot be performed until the suspicious person is found and the suspicious person is moved to the position of the suspicious person. For example, crimes by a plurality of persons cannot be dealt with.
A. Since surveillance cameras are installed at fixed positions, the entire store cannot be covered due to the number of cameras installed, blind spots of the cameras, and the like.
C. Since the suspicious person is not directly warned, the psychological pressure effect on the suspicious person is weak.
D. It is difficult to give a light warning to a criminal reserve army who commits a criminal offense without causing discomfort.

As a solution to the problem caused by the fixed-position monitoring camera, there is a method of utilizing an autonomous mobile robot that has been greatly developed in recent years. In Patent Document 1, a monitoring terminal (for example, a robot) capable of autonomous movement is connected via a communication network. In other words, a monitoring system is disclosed that transmits an image of a monitoring area captured and a collected sound to the system. However, there is no technique for moving to a suspicious person and giving a warning based on an instruction from a supervisor based on the result of monitoring the surveillance area in the store. Patent Document 2 discloses a technique in which a robot that guides an office or the like recognizes the characteristics of a guided person and autonomously moves and distinguishes them from those other than the guided person. However, there is no technique for moving to a suspicious person and giving a warning based on an instruction from a supervisor based on the result of monitoring the surveillance area in the store.
JP 2005-80100 A JP 2003-280739 A

  The problem to be solved is that there is no crime prevention system that identifies suspicious persons who may cause shoplifting and other damages at bookstores, supermarkets, etc., monitors them and alerts them while distinguishing them from customers That is.

A first aspect of the present invention is a warning method by a security robot system provided with a surveillance camera provided in a robot that performs autonomous movement, a plurality of surveillance cameras that photograph a certain area, and means for displaying an image provided in the autonomous mobile robot. der Ru.
The security robot system includes a surveillance information system, the surveillance information system accumulates videos in the store taken by the plurality of surveillance cameras, operates an operation location provided for the autonomous mobile robot, and registers a suspicious person The position of the suspicious person in the store is received based on the camera number of the surveillance camera that has photographed the suspicious person who has entered the instruction, the information of the surveillance camera, and the position pointed to by the instruction input. Calculating and registering the color of the clothes of the suspicious person inputted by the instruction and the calculated position,
For the registered suspicious person, set a patrol route around the suspicious person to the autonomous mobile robot, and from the accumulated accumulated image, the image of the image collated with the registered clothing color Extract the camera number and time, extract the suspicious person's video from the stored video based on the extracted camera number and time,
A video including the current video of the suspicious person and a video extracted from the stored video is displayed on the display means .

A second invention is a monitoring camera having a robot for performing autonomous movement, a plurality of surveillance cameras for photographing a predetermined area, and means for storing the image in the store in which the plurality of monitoring cameras have taken, the autonomous Means for operating an operation control unit provided in the mobile robot, an instruction input unit for receiving an instruction input for registering a suspicious person, a camera number of a monitoring camera that has photographed the suspicious person inputted by the instruction , information on the monitoring camera, and Means for calculating the position in the store where the suspicious person is located based on the position pointed by the instruction input unit, the color of the clothes of the suspicious person specified by the instruction input unit, and the calculated position A suspicious person registration unit for registering, a means for displaying an image provided in the autonomous mobile robot, and a periphery around the suspicious person in the autonomous mobile robot with respect to the registered suspicious person Means for setting a tour route, a video comparison / collation unit for extracting a camera number and time of a video collated with the registered clothing color from the accumulated video accumulated by the accumulating unit, and the video comparison Means for extracting the video of the suspicious person from the stored video based on the result extracted by the collating unit, and the means for displaying the video including the current video of the suspicious person and the video extracted from the stored video And a security robot system.

  According to the present invention, surveillance by a security robot at a position near the identified suspicious person, close monitoring and warning are performed step by step, causing the customer to leave the store without causing discomfort due to misidentification. It is possible to prevent and avoid store loss due to crime.

Example 1
FIG. 1 is a diagram showing a security system configuration of the present invention. It shows a system in which a video taken by a surveillance camera is edited and displayed on a display monitor, and a suspicious person monitored and identified by a supervisor is monitored and warned by a security autonomous mobile robot.

  A plurality of autonomous mobile robots 1-1, autonomous mobile robots 1-i, autonomous mobile robots 1-N, a plurality of fixed surveillance cameras 2-1, fixed surveillance cameras 2-i, fixed The monitoring camera 6 includes a monitoring camera 2-M, a communication network 3, and a monitoring information system unit 4. The monitor 6 monitors the behavior of the customer 5 in the store via the display monitor.

Each autonomous mobile robot has a mechanism for autonomous movement and the following.
A. Monitoring camera that shoots the inside of the store B. Video transmission / reception IF unit that transmits images taken by the robot to the monitoring information system unit 4 via the communication network 3 and conversely receives images from the monitoring information system unit 4 An image display unit for displaying an image instructed by the information system unit 4 d. A voice utterance unit for storing and uttering voice information for calling a customer, a suspicious person, etc. E. Operation from the monitoring information system unit 4 Operation instruction analysis / operation control unit that analyzes instructions, stores operation instructions, and gives operation instructions to controlled objects.The autonomous mobile robot autonomously circulates the specified in-store route by the above components and is instructed. Stand-by at a location in a store, autonomous patrol of a specified route, and take a picture of the surrounding video while moving autonomously. In addition, instructions are received and stored, movement / tour / standby is performed, camera control (camera position, shooting direction, shooting angle) is performed, the eyes of the autonomous mobile robot are moved, and the instructed voice is generated It has a function of storing and sequentially executing a procedure for operating each control object such as. Since these functions have many examples, they are omitted here. The autonomous mobile robot is hereinafter expressed as a robot.

  Each fixed surveillance camera is installed at a plurality of locations in the store, and transmits captured images of a certain area in the store to the monitoring information system unit 4 via the communication network.

  The communication network 3 is basically a network extended in a store, but a network via a public network is also conceivable as a monitoring center configuration for monitoring a certain area in a chain store or the like in an integrated manner.

Next, before describing the operation of the monitoring information unit 4, the processing function of each component block of the monitoring information system unit 4 will be described.
1) Video transmission unit 11: an interface for transmitting an edited video to be displayed on the video display unit of the robot to the robot via a communication network.
2) Video reception unit 12: an interface that receives video from the monitoring camera and the fixed monitoring camera mounted on the robot via the communication network 4 and transmits the received video to the video editing unit 15 and the video storage unit 13.
3) Video storage unit 13: stores the video received by the video reception unit 12. The video is accumulated and identified for each surveillance camera.
4) Video comparison / collation unit 14: A video of a specific person designated by appearance such as clothes color is searched from the stored video, and the video of the specific person collated by the color of clothes is extracted together with its camera number, shooting position, and time. . Since a method for identifying a specific person by comparing videos is known, description of the identification method based on the appearance of similarity is omitted.
5) Video editing unit 15: Real-time video from the video receiving unit 12, video content displayed on the display monitor based on past video described later from the video comparison / collation unit 16, video content displayed on the robot, real-time video and storage Selects and edits the display content of a video on one screen and the video content of switching display of multiple videos. The edited video is set by an instruction from a supervisor or a predetermined editing pattern.

The edited video is displayed on the display monitor 16, and the display instruction of the edited video is converted into a data format and transmitted to the designated robot via the video transmission unit 11 to be displayed on the specific robot designated by the monitor.
6) Display monitor 16: The edited video is displayed to the monitor.
7) Suspicious person registration unit 17: Registers the characteristics of the clothes of the suspicious person that the monitor has determined to be monitored, the color of the clothes, and the position of the suspicious person in the store at the time of the determination.
8) Instruction input unit 18: This is an interface for the monitor to input designated items to the monitoring information system unit 4. Specifically, for example, it is performed by a touch panel. The items of instructions are as follows.
A: Registration of a suspicious person a: Instruction for the route of each robot, standby instruction at a specific position c: Setting of edited video and video pattern D: Specification of a robot for monitoring a suspicious person, Video designation for robot display 9) Subject position Calculation unit 19: The camera number and information (camera position, angle of view, shooting angle) of the suspicious person photographed and inputted by the instruction input unit 18, and the suspicious person's store in the store based on the pointed position Calculate the position. Since the calculation method is a commonly used method, it is omitted.
10) Operation instruction generation unit 20: Generates operation data for operating the control target portion of the robot based on an instruction from the supervisor. Specifically, the robot's patrol route patrol instruction set by the instruction input unit 18, the stand-by instruction at a specific position, the operation instruction of the control target portion for eye movement, etc. are analyzed, and the series motion accumulating unit 22 is analyzed. Data for operating the operation control target corresponding to the analyzed operation instruction stored is extracted to generate specific operation instruction data.
11) Operation instruction transmission unit 21: The operation instruction data generated by the operation instruction generation unit 20 is converted into a data format and transmitted to the robot.
12) A series of motion accumulation units 22: accumulates information for operating motion control pair targets such as a camera mounted on the robot, a movable part for independent movement, and a movable part of the eyes. For example, information for controlling a camera position, a shooting direction, and a shooting angle, or information for operating a movable part for autonomous walking. Since it is generally performed, the description is omitted.

FIG. 2 is a diagram showing a basic operation procedure (1/2) of the security robot system of the present invention. FIG. 3 is a diagram showing a basic operation procedure (2/2) of the security robot system of the present invention. FIG. 3 and FIG. It shows the basic operation procedure from the identification of a suspicious person by a monitor in the security robot system to the patrol of the robot and the suspicious person leaving.
S1: The monitor performs the following initial settings for in-store monitoring.
A. Displays video from the designated surveillance camera on the display monitor. Here, it is also possible to perform automatic switching and display of a plurality of surveillance cameras.
A. Set a tour route in the store for the robot.
S2: The robot patrols the store, captures a video around the patrol route, and transmits it to the monitoring information system unit. The fixed monitoring camera transmits the captured in-store video to the monitoring information system unit.
S3: The monitoring information system unit displays the real-time video on the display monitor and stores it in the video storage unit for each camera. The monitor will monitor the display monitor.
S4: The monitor confirms from the displayed video whether the person needs monitoring. If there is no person who needs monitoring, the process returns to S2. If there is a suspicious person who needs monitoring, the process proceeds to S5.
S5: The supervisor registers the suspicious person as the suspicious person, and registers the camera number of the suspicious person photographing, the characteristics of the suspicious person's clothes, and the color of the clothes. The suspicious person is registered in the monitoring information system unit by the monitoring person in the suspicious person registration unit via the instruction input unit.
S6: The monitoring information system unit calculates the position of the suspicious person in the store from the camera information of the suspicious person and displays the following information on the display monitor. The position of the suspicious person in the store is calculated by the subject position calculation unit.
A: For robot camera: Robot position, robot posture, camera angle of view, camera orientation b: For fixed surveillance camera: Camera position, angle of view, shooting angle S7: At least one surveillance position for surveillance personnel Change the setting and instruct monitoring at the designated monitoring position. That is, in the case of a robot that is not patroling, a patrol route is set or moved to a specific position, and in the case of a robot that is patroling, an instruction to change the patrol path or move to a specific position is instructed. (Hereinafter, the monitoring position is expressed as a monitoring position (circulation route, specific position).) The robot that has received the instruction autonomously moves to the instructed monitoring position (circulation route, specific position).
S8: The robot sequentially transmits suspicious person monitoring and captured images to the monitoring information system unit from the instructed monitoring position (circulation route, specific position).
S9: The monitoring information system unit compares the suspicious person with the suspicious person's real-time video and the stored video, compares the real-time video with the past video, and the comparison / verification result and the related information (position where the suspicious person is, its time, shooting (Camera number) is displayed on the display monitor. Extraction of the past video of the suspicious person from the stored video will be described later.
S10: The monitor judges the monitoring position and warning method from the information on the display monitor and the current suspicious person's state, and instructs the monitoring position (circulation route, specific position), monitoring method and warning to at least one robot. Give instructions on how. The monitoring method and warning method will be described with reference to FIG.
S11: The robot moves to the instructed monitoring position (circulation route, specific position), and performs instructed monitoring and warning.
S12: The monitor determines whether the suspicious person has stopped the suspicious behavior. If it is determined that monitoring is not necessary, the process proceeds to S16, and if it is determined that the suspicious behavior continues, the process proceeds to S13 as monitoring required and continues monitoring.
S13: The robot confirms whether or not the suspicious person has left the store. If it is determined that the user has left, the process proceeds to S14. If it is determined that the user is in the store, the process proceeds to S13. Judgment as to whether or not he / she has left has been made by tracking a suspicious person and determining whether he / she has gone outside a predetermined boundary line between inside and outside.
S14: The monitor determines from the video on the display monitor that the suspicious person is still in the store, determines the monitoring position and warning method for which the level has been raised, and instructs the robot to make a change. The operation procedure returns to S11.
S15: When the warning method for the suspicious person includes automatic tracking, the monitoring person instructs the robot to end the automatic tracking, and the robot transmits a video of the suspicious person leaving the store to the monitoring information system unit. The monitor confirms that the suspicious person has left the store from the image displayed on the monitor.

For example, the tracking of the suspicious person of the robot is performed by distinguishing the suspicious person from other customers based on the similarity of the color of the clothes of the suspicious person. Since tracking of a person by a robot is generally performed, description thereof is omitted.
S16: The supervisor instructs the robot to stop monitoring / warning instructions for the suspicious person and cancels the suspicious person registration in the monitoring information system unit.

  FIG. 4 is a diagram showing an example of video editing and plane position calculation of a suspicious person in the security robot system of the present invention. A method for associating a real-time video of a suspicious person identified from a captured video with a past video is shown.

  The display image of (1) displays an image of the suspicious person in the store and at the position A in the store. (2) is an example of stored video. The position and time in the store of the suspicious person who accumulated the plurality of surveillance cameras are shown. Extraction of a video with a suspicious person in the past stored video is performed by the video comparison / collation unit of the monitoring information system unit described above.

  It shows that the fixed monitoring camera 1 is monitoring the identified suspicious person from 11:13 to the present time. The identified suspicious person's image is from 11:00 to 11:20 in the accumulated image of the fixed monitoring camera 2, from 10:25 to 10:40 in the image of the fixed monitoring camera 3, and in the robot camera 1 From 11:30 to the present time, the robot camera 3 stores images of suspicious individuals identified from 10:13 to 10:20, 10:40 to 10:50, and 11:15 to 11:25. .

  This can be interpreted, for example, as the robot camera 1 continuously monitoring from a specific position near the specific suspicious person and the robot camera 3 photographing the suspicious person on the patrol route during the patrol.

  Next, in order to extract the suspicious person's photographed image from the past accumulated image, information on the appearance such as the color of the suspicious person's clothes is input to the monitoring information system unit together with the position and time information of the photographing camera. The video comparison / verification unit evaluates the similarity of appearance such as the color of the suspicious person's clothing from the video of the fixed surveillance camera around the camera, the robot camera that circulates around the suspicious person's position, etc. Extract. Further, this method is basically the same as the method for distinguishing a suspicious person from other customers when the robot tracks the suspicious person.

  (3) shows an in-store map. The surveillance camera which image | photographed the customer 23 in a shop is specified, and the position where a suspicious person exists in a shop is converted into a position coordinate from the position of a specified camera, an angle of view, and an imaging angle, and is calculated. Description of the calculation method is omitted. Based on the calculated position of the suspicious person, the monitoring person instructs the robot to monitor the position of the suspicious person.

FIG. 5 is a diagram showing the order and procedure of the monitoring method and warning method for a suspicious person according to the present invention. The monitoring method and warning method for the suspicious person are shown together with the level. The monitoring level and the warning level are increased as the level goes down.
1) Monitoring method a: Monitoring is performed with a regular patrol route.

Each robot patrols and monitors the regular patrol route designated by the observer.
B: Monitors mainly through the patrol route centering on the position of the suspicious person.

In order to monitor a suspicious person with a specific robot, a patrol route around the suspicious person is set, and the designated robot patrolls and monitors a patrol route near the suspicious person.
C: Watch around a suspicious person with multiple robots.

A plurality of robots are set up with a route for traveling around the suspicious person, and each of the designated robots visits the designated route and monitors it.
D: Monitor at a specific location near the location of the suspicious person.

A monitoring position near the suspicious person is set for a specific robot, and the designated robot monitors the suspicious person at the designated position.
E: Track and monitor as suspicious persons move.

When the suspicious person moves, a position at a predetermined distance behind the suspicious person is detected and set to move toward the position, and the designated robot follows the suspicious person for monitoring. The back position can be detected using a distance sensor or the like. Description is omitted.
G: Monitor near the suspicious person (back, side).

The robot is set to move to the back or side of the suspicious person, and the designated robot monitors the position in contact with the designated suspicious person. The movement to the back position sets a short predetermined distance behind the suspicious person. Further, the movement to the side surface position is set so that the position detected by the distance sensor described in (e) is rotated 90 degrees clockwise or counterclockwise, and moved at a predetermined distance around the suspicious person. Description is omitted.
H: Monitor in front of the suspicious person.

The robot is set to move to a position facing the front of the suspicious person, and the designated robot monitors the position facing the designated suspicious person. The movement to the front is performed by moving to the side according to the procedure described in the key, and further by repeating the procedure described in the key.
2) Warning method a: Send a line of sight to the suspicious person from the robot.

The setting is made so that the eye attached to the robot is directed toward the face of the suspicious person, and the set robot directs the eye toward the suspicious person. Since the direction control of the parts mounted on the robot is performed, the description is omitted.
B: Respond as a customer.

Speak to the suspicious person in words that correspond to regular customers, such as “Looking for something?”
C: Display the monitor image of the shooting to the suspicious person.

The image of the suspicious person photographed from the monitoring information system unit on the display unit of a specific robot is displayed.
E: Warn the suspicious person with a voice.

Warn the suspicious person with words such as “Warning. Please stop.”
F: threaten with light.

  Intimidate the suspicious person with light.

The monitor sets the monitoring method and the warning method in order from the lowest level. Alternatively, it is performed by a method selected from the above methods based on the situation judgment.
(Appendix 1)
A plurality of surveillance cameras, including surveillance cameras for at least autonomous robots;
Means for accumulating and editing videos in a store taken by the plurality of cameras;
Means for operating an operation location provided in the autonomous mobile robot;
Means for calculating a position in the store where a specific customer is present from a video image taken by the monitoring camera;
In a security robot system equipped with a security robot in a store, a monitoring method and a warning method,
The autonomous mobile robot includes means for displaying an image,
Instructs at least one autonomous mobile robot to monitor around the suspicious person or to monitor at a specific position for a suspicious person who is determined to be monitored from the in-store video and is registered. Monitoring the suspicious person,
Displaying a video image of the suspicious person to the suspicious person and warning the suspicious person;
Monitoring method and warning method by a security robot characterized by
(Appendix 2)
A plurality of surveillance cameras, including surveillance cameras for at least autonomous robots;
Means for accumulating and editing videos in a store taken by the plurality of cameras;
Means for operating an operation control unit provided in the autonomous mobile robot;
Means for calculating a position in the store where a specific customer is present from a video image taken by the monitoring camera;
A security robot system comprising:
The autonomous mobile robot includes means for displaying an image,
For the suspicious person who is determined to be monitored from the in-store video and registered, at least one of the autonomous mobile robots is monitored around the suspicious person, or is monitored at a specific position,
A warning of displaying the video of the suspicious person on the suspicious person;
A security robot system characterized by
(Appendix 3)
The monitoring method and warning method described in Appendix 1,
The monitoring method and warning method according to appendix 1, wherein the autonomous mobile robot further includes voice utterance means, and warns the suspicious person by a method of further adding a warning or a voice threat.
(Appendix 4)
The monitoring method and warning method described in Appendix 1,
Further, the suspicious person's video is extracted from the stored video, and the suspicious person is warned by displaying a video including the current video of the suspicious person and the video extracted from the stored video. Monitoring method and warning method.
(Appendix 5)
The monitoring method and warning method described in Appendix 3,
The monitoring method and warning method according to supplementary note 3, wherein when the suspicious person does not leave the store, the monitoring method or the warning method according to supplementary note 3 is sequentially changed for monitoring and warning.
(Appendix 6)
The monitoring method and warning method described in Appendix 1,
Patrol monitoring centered on the location of the suspicious person, monitoring at a position near the suspicious person, monitoring by tracking the suspicious person, monitoring by the suspicious person, facing the suspicious person The monitoring method and warning method according to appendix 1, wherein monitoring is performed with at least one of monitoring.
(Appendix 7)
The monitoring method and warning method described in Appendix 3,
Further, the autonomous mobile robot is provided with an eye moving means or a light, and warns the suspicious person with a gaze, or warns the suspicious person with a method of further threatening the light. 3. Monitoring method and warning method according to 3.

FIG. 1 is a diagram showing a security system configuration of the present invention. FIG. 2 is a diagram showing a basic operation procedure (1/2) of the security robot system of the present invention. FIG. 3 is a diagram showing a basic operation procedure (2/2) of the security robot system of the present invention. FIG. 4 is a diagram showing an example of video editing and plane position calculation of a suspicious person in the security robot system of the present invention. FIG. 5 is a diagram showing the order and procedure of the monitoring method and warning method for a suspicious person according to the present invention.

Explanation of symbols

1-1, 1-i, 1-N Autonomous mobile robots 2-1, 2-i, 2-M Fixed monitoring camera 3 Communication network 4 Monitoring information system unit 5 Customer 6 Monitor 11 Video transmission unit 12 Video reception unit 13 Video storage unit 14 Video comparison / collation unit 15 Video editing unit 16 Display monitor 17 Suspicious person registration unit 18 Instruction input unit 19 Subject position calculation unit 20 Operation instruction generation unit 21 Operation instruction transmission unit 22 Continuous operation storage unit 23 Suspicious person

Claims (2)

A surveillance camera for a robot that moves autonomously;
Multiple surveillance cameras that capture a certain area;
A warning method by a security robot system provided on the autonomous mobile robot and provided with means for displaying an image,
The security robot system includes a monitoring information system;
The monitoring information system includes:
Accumulating in-store images taken by the plurality of surveillance cameras,
Operate the operation part prepared for the autonomous mobile robot,
Accepts instructions to register suspicious individuals,
Calculate the position in the store where the suspicious person is located based on the camera number of the surveillance camera that has photographed the suspicious person who has been inputted as an instruction, the information of the surveillance camera and the position pointed to by the instruction input,
Registering the color of the clothes of the suspicious person inputted as input and the calculated position,
For the registered suspicious person, set a patrol route around the suspicious person in the autonomous mobile robot,
Extracting the camera number and time of the video collated with the registered clothing color from the accumulated video,
Extracting the suspicious person's video from the stored video based on the extracted camera number and time,
A warning method by a security robot system, characterized in that a video including a current video of the suspicious person and a video extracted from the stored video is displayed on the display means. A surveillance camera for a robot that moves autonomously;
Multiple surveillance cameras that capture a certain area;
Means for accumulating in-store images taken by the plurality of surveillance cameras;
Means for operating an operation control unit provided in the autonomous mobile robot;
An instruction input unit for receiving an instruction input for registering a suspicious person;
Means for calculating a position in the store where the suspicious person is located based on the camera number of the surveillance camera that has photographed the suspicious person who has been inputted as an instruction, the information of the surveillance camera, and the position pointed by the instruction input unit;
A suspicious person registration unit that registers the color of the clothes of the suspicious person specified in the instruction input unit and the calculated position;
Means for displaying an image provided in the autonomous mobile robot;
Means for setting a patrol route around the suspicious person in the autonomous mobile robot for the registered suspicious person;
A video comparison and collation unit for extracting the camera number and time of the video collated with the registered clothing color from the accumulated video accumulated by the accumulating unit;
Means for extracting the video of the suspicious person from the stored video based on the result extracted by the video comparison and collation unit,
A security robot system characterized in that an image including a current image of the suspicious person and an image extracted from the accumulated image is displayed on the display means.

Images