JP2012078950A - Monitoring system with autonomous mobile body, monitoring device, autonomous mobile body, monitoring method and monitoring program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To patrol through an appropriate route by using a characteristic retrieval function.SOLUTION: In a monitoring system comprising at least one autonomous mobile body patrolling through a prescribed route in a monitoring zone and a monitoring device for monitoring the monitoring zone by using the autonomous mobile body, the monitoring device includes specific person extraction means for extracting a specific person by using an image imaged by at least one of imaging means installed in the monitoring zone, important monitoring point selection means for selecting an important monitoring point from location information on the imaging means which imaged the image acquired by the specific person extraction means, and patrol instruction means for generating patrol request information corresponding to the point acquired by the important monitoring point selection means and instructing the autonomous mobile body on the generated patrol request information. The autonomous mobile body has patrol route setting means for setting a patrol route by using the patrol request information acquired from the monitoring device and the preset patrol route.

Description

  The present invention relates to a monitoring system, a monitoring device, an autonomous moving body, a monitoring method, and a monitoring program using an autonomous moving body, and more particularly, to an autonomous moving body for performing patrol by an appropriate moving route using a feature search function. The present invention relates to a monitoring system, a monitoring device, an autonomous mobile body, a monitoring method, and a monitoring program used.

  2. Description of the Related Art Conventionally, there is a method of performing image processing or the like on a video taken by an imaging unit such as a camera and automatically detecting a person included in an image constituting the video. Among them, a technique for detecting a human face from an image has been rapidly developed.For example, in order to improve face discrimination accuracy, a skin color region is extracted from an input image, and based on the skin color region, the top of the head, There is an image processing apparatus that detects the positions of facial feature points such as eyes and mouth, and determines whether or not the skin color region is a face from the detection result (see, for example, Patent Document 1).

  In addition, such face detection technology is applied to images obtained by surveillance cameras installed in surveillance areas such as banks, department stores, and convenience stores. It is used to prevent crimes by detecting whether or not.

  Furthermore, there is a method for searching for a face image having a specified feature by accumulating detected face information together with feature information and specifying a facial feature of a person to be searched as a search condition (for example, Patent Document 2).

  It should be noted that the above-described person identification information has been increasingly used for security purposes, mainly by identifying suspicious persons.

JP 2004-5384 A JP 2006-318375 A

  By the way, a suspicious person or the like detected by the above-described conventional method needs to immediately notify a guard or the like. This is the same for an autonomous moving body such as a surveillance robot for security, as well as for humans.

  However, when the surveillance robot patrols a predetermined surveillance area, it patrols according to a route set in advance at a preset time. Therefore, it is necessary to set a tour route in advance before the tour starts. Therefore, if a suspicious person or a lost child is discovered on the monitoring center side during the patrol of the monitoring robot, the monitoring robot is notified of specific information of the person, the patrol route is changed, the patrol pattern is changed, etc. There was no monitoring system using an autonomous mobile body that could respond flexibly.

  The present invention has been made in view of the above-described problems, and includes a monitoring system, a monitoring device, an autonomous moving body using an autonomous moving body for performing patrol by an appropriate moving route using a feature search function, It is an object to provide a monitoring method and a monitoring program.

  In order to solve the above problems, the present invention employs means for solving the problems having the following characteristics.

  In the present invention, in the monitoring system comprising at least one autonomous mobile body that patrols the monitoring area along a predetermined route and a monitoring device that monitors the monitoring area using the autonomous mobile body, the monitoring device includes the monitoring device. It is important from the position information of the specific person extracting means for extracting a specific person using an image taken by at least one imaging means installed in the area, and the imaging means for taking the image obtained by the specific person extracting means. An important monitoring point selecting means for selecting a monitoring point; and a traveling instruction means for generating traveling request information corresponding to the point obtained by the important monitoring point selecting means and instructing the generated traveling request information to the autonomous mobile body; And the autonomous mobile body sets a cyclic route using the cyclic request information obtained from the monitoring device and a preset cyclic route. Characterized in that it has a means.

  In the present invention, the important monitoring point selecting means calculates a score for each of a plurality of places preset in the monitoring area.

  In the present invention, the patrol request information includes the important monitoring point and a priority for each important monitoring point.

  In the present invention, when the autonomous mobile body is traveling, the traveling route setting means obtains current location information, and based on the distance between the obtained location information and the important monitoring point, Is set.

  According to the present invention, in the monitoring device that monitors the monitoring area using at least one autonomous mobile body that patrols the monitoring area along a predetermined route, an image taken by at least one imaging unit installed in the monitoring area A specific person extracting unit for extracting a specific person, an important monitoring point selecting unit for selecting an important monitoring point from position information of an imaging unit that has taken an image obtained by the specific person extracting unit, and the important monitoring It is characterized by comprising: patrol requesting means for generating patrol request information corresponding to the spot obtained by the spot selecting means and instructing the generated patrol request information to the autonomous mobile body.

  In the present invention, the important monitoring point selecting means calculates a score for each of a plurality of places preset in the monitoring area.

  In the present invention, the patrol request information includes the important monitoring point and a priority for each important monitoring point.

  Further, in the present invention, in an autonomous mobile body that patrols a monitoring area on a predetermined route according to an instruction from the monitoring device, the traveling route is determined using the tour request information obtained from the monitoring device and a preset traveling route. A traveling route setting means for setting; a moving means for moving based on the traveling route obtained by the traveling route setting means; and an imaging means for capturing an image of surroundings moving along the traveling route. .

  In the present invention, when the autonomous mobile body is traveling, the traveling route setting means obtains current location information, and based on the distance between the obtained location information and the important monitoring point, Is set.

  According to the present invention, there is provided a monitoring method in a monitoring system comprising at least one autonomous mobile body that patrols a monitoring area on a predetermined route, and a monitoring device that monitors the monitoring area using the autonomous mobile body. The specific person extraction step of extracting a specific person using an image taken by at least one image pickup means installed in the monitoring area, and the image pickup means for picking up the image obtained by the specific person extraction step An important monitoring point selection step for selecting an important monitoring point from position information, and tour request information corresponding to the point obtained by the important monitoring point selection step are generated, and the generated tour request information is instructed to the autonomous mobile body A patrol instruction step, patrol request information obtained from the monitoring device by the autonomous mobile body, and a patrol route set in advance. With bets, characterized by having a cyclic path setting step of setting a patrol route.

  In the present invention, the important monitoring point selection step calculates a score for each of a plurality of places preset in the monitoring area.

  In the present invention, the patrol request information includes the important monitoring point and a priority for each important monitoring point.

  Further, in the present invention, when the autonomous mobile body is traveling, the traveling route setting step acquires current location information, and based on the distance between the obtained location information and the important monitoring point, the traveling route Is set.

  The present invention also provides a monitoring program that causes a computer to function as the monitoring device according to any one of claims 5 to 7.

  Moreover, this invention is a monitoring program characterized by making a computer function as an autonomous mobile body of Claim 8 or 9.

  According to the present invention, it is possible to perform patrol with an appropriate travel route using the feature search function.

It is a figure which shows an example of the system outline | summary of the movement management system of an autonomous mobile body. It is a figure which shows an example of a function structure of the monitoring apparatus in this embodiment. It is a figure which shows an example of a function structure of the monitoring robot in this embodiment. It is a figure for demonstrating the outline | summary until it acquires the person characteristic for a search in this embodiment. It is a figure of an example for demonstrating the search method in this embodiment. It is a sequence chart which shows an example of the operation processing procedure at the time of the real-time search in this embodiment. It is a sequence chart which shows an example of the operation processing procedure at the time of a suspicious part patrol. It is a sequence chart which shows an example of the operation | movement process procedure at the time of the process after the designated characteristic person is extracted. It is a sequence chart which shows an example of the operation | movement process sequence at the time of the cyclic route setting by the monitoring apparatus side. It is a sequence chart which shows an example of the operation | movement process sequence when providing the search function in the monitoring robot. It is a figure which shows the example of a screen transition in this embodiment. It is a figure which shows the example of a screen of the robot path | route setting in this embodiment.

<About the present invention>
The present invention provides a security service and a facility monitoring service by operating a monitoring center (monitoring device) having a function of searching for a feature of a person from a video or an image and an autonomous mobile body such as a monitoring robot in cooperation with each other. It is intended to reduce the work load in Japan and prevent crimes in manned environments.

  Specifically, in the present invention, for example, the monitoring center side acquires an image of a camera installed in the monitoring area, performs image analysis, creation of a search tag, etc., and the robot side generates and generates a patrol route. Performs a patrol operation along the patrol route.

  Further, in the present invention, when a search request for a person having a feature specified using a GUI (Graphical User Interface) or the like is made on the monitoring center side, the feature search side uses the specified feature as a search key from the stored image. The corresponding person is searched, and from the result, one or a plurality of monitoring robots are notified as points having a high number of searched people candidates or a point where a person close to the specified feature is detected as a priority patrol point. In addition, the monitoring robot side creates a route for efficiently patroling the priority patrol point and performs the patrol. As a result, the present invention can more efficiently search for a designated person.

  In the present invention, the monitoring center side has a function of performing a suspicious search every set time and calculating the suspicious degree based on the number of persons determined to be suspicious at each point and the search score. That is, when a point with a high suspicious degree appears, the monitoring robot is notified as a priority patrol point and a patrol request is made. Furthermore, the surveillance robot makes a route by creating a route for efficiently visiting the priority patrol points. As a result, places where many suspicious persons have been confirmed are circulated, and crime can be prevented.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments in which a monitoring system, a monitoring device, an autonomous moving body, a monitoring method, and a monitoring program using an autonomous mobile body according to the present invention are suitably implemented will be described with reference to the drawings.

<System overview example of a monitoring system using autonomous mobile objects>
Here, a system outline example of a monitoring system using an autonomous mobile body will be described with reference to the drawings. FIG. 1 is a diagram illustrating an example of a system outline of a mobility management system for an autonomous mobile body. A mobility management system 10 shown in FIG. 1 is provided in a monitoring camera 11-1, 11-2,..., 12-n as imaging means set in a monitoring facility set in advance, and the monitoring facility. Each sensor group 12, a monitoring device 13, and a plurality of monitoring robots 14-1, 14-2,..., 14-n, etc. as autonomous moving bodies that patrol a preset monitoring facility. Has been.

  The monitoring camera 11, each sensor group 12, the monitoring device 13, and the monitoring robot 14 are connected to each other in a state where data can be transmitted and received through a communication network 15 represented by the Internet or the like. The communication network 15 may be wired communication or wireless communication.

  Each of the plurality of monitoring cameras 11-1 to 11-n is provided with an identification number, and an identification number, a shooting date and time, and the like are added to each video obtained from the monitoring camera 11. That is, the monitoring device 13 that has acquired the video from the monitoring camera 11 can easily grasp where the monitoring camera is installed from the identification information added to the video. Moreover, the monitoring camera 11 may be provided with the audio | voice input apparatus which records the surrounding audio | voice, and may transmit the audio | voice signal corresponding to a video signal in that case.

  Each sensor group 12 is a security or security sensor provided in a monitoring area. For example, an infrared sensor for detecting an intruder by infrared rays or a fire for detecting fire, heat, smoke, or the like. It has a sensor, a gas sensor that detects gas leakage, an open / close detection sensor that detects opening and closing of windows and doors, a breakage detection sensor that detects breakage of windows and fences, and the like. When each sensor group 12 detects a corresponding condition, it outputs detection information indicating the detection to the monitoring device 13 via the communication network 15. Each sensor group 12 is set with identification information for identifying each sensor, and the detection information described above includes the identification information, date information, and the like. Thereby, the monitoring apparatus 13 can grasp | ascertain easily where each sensor is set to the monitoring object facility or the monitoring object area by acquiring the detection information mentioned above.

  The monitoring device 13 is a management device that manages the traveling routes, the security status, and the like of the plurality of monitoring robots 14-1, 14-2, ..., 14-n. The monitoring device 13 sets or changes a security patrol route for at least one monitoring robot among the monitoring robots 14-1 to 14-n based on video and detection signals obtained from the monitoring camera 11 and each sensor group 12. To do.

  Here, the circulation route is, for example, not only the order of each set point or area (for example, A point, B point, C point, D point, etc.), but also the number of visits to the point or area, the traveling time, and the circulation. Contents (for example, monitoring only the front, monitoring the entire surroundings, etc.) are also included.

  In addition, the monitoring device 13 uses two or more monitoring robots to adjust each patrol route, and for example, a suspicious person performed using a detected sensor installation position or a video signal obtained from the security camera 11. When searching for a specific person, such as a child or lost child, based on the installation position of the camera etc. that shot the video that was determined to be a specific person, set the patrol route so that the point and its surroundings are monitored intensively .

  Moreover, the monitoring robots 14-1 to 14-n guard the corresponding points based on the patrol instruction signal from the monitoring device 13. The surveillance robot has a voice output function, a recording function, a photographing function, a screen display function, and the like, and can input or output voice or video on the spot.

<Monitoring device 13: functional configuration example>
Next, a functional configuration example of the monitoring device 13 described above will be specifically described with reference to the drawings. FIG. 2 is a diagram illustrating an example of a functional configuration of the monitoring apparatus according to the present embodiment.

  2 includes an input means 21, an output means 22, a storage means 23, a face / human body region detection means 24, an identification means 25, a tracking means 26, a suspicious person detection means 27, Person information integration means 28, frame information generation means 29, search means 30, important monitoring point selection means 31, tour instruction means 32, screen generation means 33, notification means 34, transmission / reception means 35, and control Means 36.

  The face / human body region detection means 24, identification means 25, tracking means 26, suspicious person detection means 27, person information integration means 28, frame information generation means 29, and search means 30 described above are specified in the present embodiment. It is included in the function as a person extraction means.

  The input means 21 is a face / human body region detection instruction from a user, identification instruction, tracking instruction, suspicious person detection instruction, person information integration instruction, frame information generation instruction, search instruction, important monitoring point selection instruction, patrol instruction, Various instructions such as a screen generation instruction, a notification instruction, and a transmission / reception instruction are accepted. Note that the input unit 21 includes, for example, a keyboard, a pointing device such as a mouse, a voice input device such as a microphone, and the like.

  The output means 22 displays various information such as the progress or result executed in each configuration by the instruction content input by the input means 21 and the control data generated based on each instruction content, and the sound. Output. The output means 22 has a screen display function such as a display, a sound output function such as a speaker, and the like.

  Further, the output unit 22 outputs the search result obtained by the search unit 30 and the information displayed on the screen generated by the screen generation unit 33 to the external device. That is, the output unit 22 outputs, for example, to a printer as an output to an external device, generates a file and outputs the file to a recording device or a recording medium such as a database, or the monitoring robot 14 based on a search result. Outputs the generated control signal, switches on / off the sensor in the monitoring area (facility to be guarded), turns on / off the light, and related information based on the search result for the mobile terminal possessed by the guard It has a print / output function and the like for outputting a control signal for displaying (location and contents where there is an abnormality). Moreover, the output means 22 can output simultaneously to the 1 or several external apparatus mentioned above.

  The storage unit 23 can store various information for realizing the above-described embodiment, and reading and writing are performed as necessary. Specifically, the accumulating unit 23 performs various feature amount data used for face authentication, gender and age estimation, face area detection results and human body region detection results in the face / human body region detection unit 24. The person identification result in the identification means 25, the tracking result in the tracking means 26, the suspicious person detection result in the suspicious person detection means 27, the person information integration result in the person information integration means 28, the frame information generation result in the frame information generation means 29, and the search A search result in the means 30, an important monitoring point selection result in the important monitoring point selection means 31, a tour instruction result in the tour instruction means 32, a screen generation result in the screen generation means, a notification result in the notification means 34, transmission / reception information in the transmission / reception means 35, Information controlled by the control means 36, error information at the time of error occurrence, log information, Each information programs for current are accumulated.

  The face / human body region detection unit 24 converts the above-described monitoring camera 11-1 to 11-n into a real-time image captured by the imaging unit 26 and the like, and a huge amount of monitoring video already stored in the storage unit 23. On the other hand, when it is determined that a person is included in the image in the video, the face area of the person is detected. That is, the face / human body region detection unit 24 acquires the video imaged by the monitoring camera 11 or the like via the transmission / reception unit 35, and among the time-series images included in the acquired video image, a predetermined image ( The face of one or a plurality of persons is detected for each frame image or an image with an interval of several frames.

  Specifically, the face / human body region detection unit 24 acquires the feature amount of the face from position information such as eyes, nose, and mouth in the face included in the photographed image, and detects it as a preset face. The face of a person is detected by performing a matching process using a matching pattern of feature amounts. Further, the face / human body region detection means 24 is not limited to the above-described face detection processing, and for example, face detection by edge detection or shape pattern detection, face detection by hue extraction or skin color extraction, or the like can be used. The face / human body region detection means 24 detects a rectangular face region having a vertical width and a horizontal width of the face in the image after the face detection.

  Further, the face / human body region detection means 24 detects the center coordinates (position information) of the face region and the size (size) of the region on the image, and synthesizes the face region into the original image with a predetermined shape. Various information for screen display is acquired so that the face area can be clearly understood, and is stored in the storage means 23. In the present invention, the shape of the face region may be a rectangle, a circle, an ellipse, another polygon, a silhouette shape enlarged from the outer shape of a human face at a predetermined magnification, or the like.

  In addition, the face / human body region detection unit 24 applies a video to a real time video captured by each of the monitoring cameras 11-1 to 11-n or the like and a huge amount of the monitoring video already stored in the storage unit 23. When it is determined that a person is included in the inside image, the human body region of the person is detected. In other words, the face / human body region detection unit 24 acquires videos captured by the monitoring cameras 11-1 to 11-n and the like via the transmission / reception unit 35, and each time-series image included in the acquired video. Among them, one or a plurality of human body regions are detected for a predetermined image (such as each frame image or an image having an interval of several frames).

  Specifically, the face / human body region detection unit 24 compares, for example, successive image frames, and there is a place where the color information (luminance, chromaticity, etc.) changes within a predetermined time. An area surrounded by a predetermined area or more, or a movement range with time within a predetermined range is detected as a human body area. It should be noted that the human body detection method is not limited to this in the present invention.

  Further, the face / human body region detecting means 24 detects the center coordinates of the human body region and the size of the human body region on the image, and synthesizes the human body region with the original image with a predetermined shape to clearly understand the human body region. As described above, various information for screen display is acquired and stored in the storage unit 23. The shape of the human body region may be a rectangle, a circle, an ellipse, another polygon, a silhouette shape enlarged at a predetermined magnification from the outer shape of a person, and the like, similar to the face region described above.

  Further, as an additional function, the face / human body region detection unit 24 performs person tracking using the detected face region, and at that time, in order to identify and accumulate each of the plurality of face regions included in the image frame. Assign identification information (tracking ID), perform face authentication processing, output identification information (ID) and person name of the authenticated person, estimate gender and age, face area is mask, sunglasses, etc. It may have a function of detecting whether the person is hidden or not, or estimating the height of the person from the relationship between the person and the building in the image.

  Further, the face / person detection unit 24 may have a function of extracting color information such as hair, upper garment, and lower garment, and calculating position coordinates of the person in the real space.

  The identification unit 25 performs identification processing for determining whether or not the same person is included in one or a plurality of persons detected from two images extracted from videos taken at different times with the same camera. . Specifically, for example, the identification unit 25 determines whether the person included in the shooting is a person who has been shot or a new person, for example, whether or not there is a person in the previous video If there is a person, it is determined whether or not the person is included in a preset movable range, and further, the same person is determined by the presence or absence of similarity based on comparison between the front and back face feature amounts. Can be determined.

  The identification unit 25 is similar to the face pattern detected by the face / human body region detection unit 24 and personal information (name, gender, age, and celebrity (person) stored in the storage unit 23 in advance. Etc.) can be compared to determine the face of the person, that is, who the person is. Furthermore, the identification means 25 can also acquire a similarity such as how much the face is similar to whom. As a result, the screen generating unit 33 and the like can display a predetermined number of images in which persons with high similarity to a certain person are displayed in descending order of similarity.

  Note that the identification unit 25 may identify a person using an appropriate parameter by a method of selecting a feature amount according to the state of the image. For example, the identification unit 25 determines a person's state (stopped, walking, whether there was a change in sunshine, etc.), and based on the result, various feature amounts (gait (step length, walking speed, posture, etc.) are determined. Including), body type, gender, clothes color, hair color, etc.) can be used for person identification. Further, the identification unit 25 calculates the degree of separation only for the feature amount to be used, performs weighting based on the degree of separation, and changes the contribution rate for each feature amount with respect to person identification. Thereby, highly accurate person determination can be performed using the feature-value according to a condition.

  The tracking unit 26 tracks the behavior of the same person identified by the identifying unit 25 over time (in time series) using the captured video. The tracking unit 26 can track a plurality of persons included in the video over time in parallel. In addition, when tracking the person included in the video over time, the tracking unit 26 may estimate the next movable range from the orientation, posture, action so far, and the like of the person acquired from the image. it can. In this case, the tracking unit 26 can generate information for combining the transferable range with the image and output the information to the screen generation unit 33.

  Here, the tracking means 26 tracks the person from the position of the foot and head of the human body region included in the image and its characteristics, and when the size of the person that can be determined to be the same changes, It is judged whether or not it is hidden by etc. The part of the person to be concealed is, for example, the lower body, the upper body, the head, arms, and the torso. In other words, in this embodiment, when information about the height of the human body from the toes of the feet to the head can be acquired at least once on the screen, if both the head and feet are not simultaneously hidden, The person can be estimated and tracked.

  The suspicious person detecting means 27 compares the result of tracking by the tracking means 26 with preset suspicious behavior patterns, behavior data, etc. stored in the accumulating means 23, and is taken by the surveillance camera 11. Or, a person corresponding to a suspicious person among a plurality of persons is detected.

  In other words, the suspicious person detecting means 27 has at least one suspicious action such as the person to be tracked hidden behind the shield, the camera being looked at for a predetermined time or longer, or being swayed or staying for a long time. In that case, the person is detected as a suspicious person. Further, the suspicious person detecting means 27 detects a suspicious person even when the person to be tracked is wearing a mask or wearing sunglasses to hide his / her face.

  Further, when the suspicious person detecting means 27 detects the suspicious person, the suspicious person detecting means 27 accumulates all the contents including information on which suspicious action is applicable in the accumulating means 23. As a result, when “what corresponds to a suspicious person” is designated as a keyword at the time of search, it is easily extracted by a simple operation of the searcher. Further, the suspicious person detecting means 27 may generate a screen for notifying the screen generating means 33 of the contents to that effect when the suspicious person is detected.

  The person information integration unit 28 associates the face area and the human body area as the same person, and integrates the characteristics of the person. Specifically, the person information integration unit 28 acquires the center of gravity coordinates of the face area in the image, and when there is a human body area that includes the acquired coordinates, the face area and the human body area are from the same person. Associating as there is.

  Furthermore, when one human body region includes two or more face regions, the person information integration unit 28 associates the same human body region with all the face regions. Thereby, even when a plurality of persons temporarily overlap on the screen during the tracking process or the like, it can be continuously tracked without interruption. Persons are managed using identification information such as IDs. Note that the processing for associating the human body region and the face region as those of the same person is not limited to this in the present invention. For example, the posture and orientation of the person are extracted and the extracted information is used. May be associated with each other.

  The frame information generation means 29 generates frame information in which the person information for each person integrated by the person information integration means 28 described above is stored for each frame of the image included in the video, and the generated search person feature data Is stored in the storage means 23 as a database or the like. Thereby, the characteristics of a person can be managed in units of frames. In addition, since the time information of the shot is given for each frame, various searches can be performed with high accuracy using various search keywords such as what kind of person at what time during the search. can do.

  The frame information in the present embodiment includes, for example, “file name” and “detection date / time” as common items, and “person position coordinates (X, Y, Z)” as information extracted from the face area, “Height information”, “various color information”, and “registrant information”, “similar celebrity information”, “age information” when already extracted people are registered in a preset database, etc. “Gender information”, “Face orientation (PAN, TILT)”, “Mask presence / absence”, “Sunglasses presence / absence”, “Acquired face image information”, etc. In addition, “person position coordinates (X, Y, Z) "," height information ", etc., but the present invention is not limited to this.

  The search means 30 is used by the searcher according to the present embodiment for various kinds of information such as real-time videos taken by the respective monitoring cameras 11-1 to 11-n and the like and monitoring videos already stored in the storage means 23. A predetermined selectable keyword to be specified, an extraction condition or the like is set, or a detection condition for detecting a suspicious person or a lost child is set in advance, and a search result such as an image including a corresponding person is extracted.

  The search means 30 automatically sets features such as a face and clothes (including not only clothes but also hats, masks, glasses, etc.), action extraction conditions, etc. for the search results, and the search results. It is also possible to perform a similar search for searching for a person similar to, and output the similarity, or output a predetermined number of search results in descending order of similarity.

  In addition, when the search unit 30 searches for a lost child or the like in real time using the above-described monitoring robots 14-1 to 14-n, the search unit 30 applies in advance to an image from the imaging unit provided in the monitoring robot. It also has a function of searching for an image on which a person satisfying the set conditions (height, clothes color, location information, etc.) is displayed and displaying the result in real time by the screen generation means 33.

  The search screen in the search unit 30 and the output result displayed as a result thereof are generated by the screen generation unit 33 and output by the output unit 22. Thereby, in this embodiment, the search which satisfy | fills a predetermined condition can be performed based on the image | video and the real-time image | video accumulated beforehand, and the search result can be displayed.

  The important monitoring point selection means 31 selects, as a critical monitoring point, a point where the person was or a point around it for a specific person such as a suspicious person or a search target person such as a lost child obtained by the search means 30 or the like. . In addition, the information of the place where the specific person is searched is acquired from, for example, the identification information for each monitoring camera 11 included in the captured video and the installation position information associated with the identification information of the monitoring camera 11 accumulated in advance. be able to. In the present embodiment, installation position information may be included in the identification information. In that case, the installation position information can be directly acquired from the identification information.

  Moreover, the important monitoring point selection means 31 selects a priority patrol point on the basis of the important monitoring point described above. That is, the important monitoring point selection unit 31 extracts the extraction score (similarity) of the specific person for each point (location) set in advance in the monitoring area, the number of times the important monitoring point is selected, the selected time, and the predetermined time. The location-specific score is calculated using at least one of the selection frequencies and the like, the candidate points for important monitoring to be preferentially visited are selected from the result, and the priority composed of the location-specific scores is output.

  The patrol instruction unit 32 takes charge of patrol of the point among the plurality of monitoring robots 14-1 to 14-n in order to perform priority patrol for the important monitoring point selected by the important monitoring point selecting unit 31. A patrol request is generated for one or a plurality of monitoring robots, and a patrol instruction based on the generated patrol request information is transmitted to the target monitoring robot.

  The screen generation means 33 is a video image taken by the monitoring camera 11, a face area detected by the face / human body area detection means 24, a human body area detected by the face / human body area detection means 24, or a person search in this embodiment. A screen for outputting a notification result or the like in the notification means 34, such as a menu screen for performing the search, an input screen used in the search by the search means 30, or a search result screen, is generated. At this time, the screen generation unit 33 can display, for example, digitized data such as position information corresponding to the region of the photographed person.

  The screen generation means 33 generates and displays a screen so that various data such as a movable range used when tracking a person included in the video and a tracking route obtained by the tracking means 26 can be easily monitored. Can do. Furthermore, when the screen generation unit 33 corresponds to a preset suspicious person's behavior pattern, the screen generation unit 33 generates a screen for notifying the content to that effect.

  Further, when displaying the search result obtained by the search unit 30, the screen generation unit 33 may temporarily stop the update display of the search result when displaying the search result in real time. it can. Thereby, for example, when the search result is updated and displayed as needed in real time, the necessary information can be displayed without forgetting to confirm the information by pausing. Further, the screen generation means 33 can continuously display the display content suspended as described above.

  Further, the screen generation means 33 not only displays the characteristics or actions of the person included in the video, but, for example, when one or more monitoring robots provided with imaging means are traveling around the monitoring area. It is also possible to display the patrol position information (three-dimensional coordinates, latitude, longitude, etc.), the patrol route, etc. transmitted together with the monitoring video for each monitoring robot. Thereby, the current position for each monitoring robot can be accurately grasped.

  Various kinds of information necessary for screen generation can be used by appropriately reading out necessary information from information stored in the storage unit 23 in advance. Further, the screen generation unit 33 can display the generated screen or the like on a display as the output unit 22 or output sound or the like through a speaker or the like.

  The notification means 34 generates an image detected as a suspicious person obtained by the suspicious person detection means 27 and information related to the image (detection date and time, detection location, video for a predetermined time, etc.) by the screen generation means 33. And displayed by the output means 22. In addition, the notification means 34 is an alert that notifies the security officer, supervisor, representative representative, surveillance robot, etc. in charge of the building in the manager or security company in response to the problem occurrence signal in such suspicious behavior detection. It has a function.

  For example, when notifying the monitoring robot or the like, the notification means 34 faces the target person or is close enough to the target person to be photographed by the imaging means of the monitoring robot. In addition, a voice signal can be output from the monitoring robot to the target person, or a control signal can be notified to the monitoring robot 14 that causes a warning lamp, emergency sound, or the like to perform a process of calling attention to the surroundings. .

  The transmission / reception means 35 is connected to an imaging means provided in one or a plurality of monitoring cameras 11-1 to 11-n and monitoring robots 14-1 to 14-n via a communication network such as a LAN (Local Area Network) or the Internet. Receive surveillance video. The transmission / reception means 35 does not have to receive the monitoring video directly from the monitoring camera 11. For example, the transmission / reception means 35 temporarily stores the video acquired in advance by the monitoring camera 11 somewhere and uses the stored information. In this embodiment, human body detection may be performed.

  Further, the transmission / reception means 35 can be used as a communication interface for transmitting to other terminals constituting the monitoring device 13 and receiving various data from other terminals.

  The control unit 36 controls the entire functional configuration in the monitoring device 13. Specifically, the control unit 36 performs various controls such as executing processing in each functional configuration described above based on input information from the user input by the input unit 21.

  Further, for example, when the monitoring robot 14 is traveling around the monitoring area, the control unit 36 controls the monitoring robot 14 regarding the traveling route, the operation when the target person is found, and the like. Specifically, the control unit 36 causes the notification unit 34 to output a voice message, a warning, or the like for the target person and its surroundings when the target person is discovered, or forcibly ends the patrol of the monitoring robot 14 after the target person is found. Control such as setting a return path is performed. In other words, when the monitoring robot 14 finds the target person, the patrol can be ended and the route can be set so that the target person is guided to a predetermined place such as a management room.

  Further, the control means 36 generates a control signal that causes the monitoring robot 14 to perform an emergency operation in response to a warning signal (for example, fire, water leakage, person detection, etc.) obtained from the monitoring robot 14. A control signal can be transmitted to the monitoring robot 14 or the like via the output means 22 or the transmission / reception means 35 or the like to control the monitoring robot 14.

<Monitoring robot 14: functional configuration example>
Next, a functional configuration example of the monitoring robot 14 described above will be specifically described with reference to the drawings. FIG. 3 is a diagram illustrating an example of a functional configuration of the monitoring robot according to the present embodiment.

  3 includes an input unit 41, an output unit 42, a storage unit 43, a moving unit 44, a patrol route setting unit 45, an imaging unit 46, a tracking / guidance unit 47, and a moving body. An interlinkage unit 48, a notification unit 49, a transmission / reception unit 50, and a control unit 51 are included.

  The input unit 41 receives input instructions from the monitoring device 13 and instruction information input manually by various operation buttons and a touch panel provided on the monitoring robot 14. The input unit 41 may be a voice input device such as a microphone.

  The output means 42 displays various information such as the progress or result executed in each configuration by the instruction content input by the input means 41 and the control data generated based on each instruction content, Output. The output means 42 has a screen display function such as a display and a touch panel, a voice output function such as a speaker, and the like.

  The storage unit 43 can store various information for realizing the above-described embodiment, and reading and writing are performed as necessary. Specifically, the accumulating unit 43 is configured so that the traveling route set in the traveling route setting unit 45, the video and audio in the imaging unit 46, the progress or result of tracking or guidance in the tracking / guidance unit 47, and the inter-movement cooperation unit 48 Information such as inter-movement cooperation results, notification results in the notification means 49, transmission / reception information in the transmission / reception means 50, information controlled by the control means 51, error information at the time of error occurrence, log information, programs for realizing the present invention, etc. Is accumulated.

  Furthermore, the accumulating unit 43 registers specific person information such as suspicious person information and lost child information obtained by the monitoring device 13 and is used when the tracking / guidance unit 47 collates the person.

  The moving means 44 has a moving function using a tire for moving the monitoring robot 14, a walking function, and the like, and moves based on a predetermined point or a patrol route.

  The patrol route setting means 45 is based on patrol request information based on suspicious person information, lost child information, or the like newly obtained by the monitoring device 13 based on the patrol route previously assigned to each monitoring robot 14 by the monitoring device 13 or the like. Change the patrol route.

  In addition, when the monitoring robot 14 is patroling, the patrol route setting unit 45 temporarily stops patrol and acquires the current position information, and the distance between the acquired position information and the important monitoring point and the important monitoring point Based on the priority and the route information ahead of the present time among the preset cyclic routes, a new cyclic route is reset.

  Thereby, it is possible to perform the patrol using the new patrol route from the continuation that has been patroling until now. Also, if you are traveling around a point that has not yet been visited as an important monitoring point, when traveling around that point, you can slow down the traveling speed from the default speed or stop the movement. It is possible to perform processing such as increasing the time at the point from the predetermined time, and if the point has already been patroled, perform processing such as patroling the point again after the predetermined patrol route is over, The patrol guard can be dealt with on an as-needed basis.

  The imaging unit 46 captures images of the front and surroundings of the monitoring robot 14 with a camera or the like provided on the monitoring robot 14. Note that the imaging unit 46 may have a voice input function for inputting surrounding voices. In addition, the video and audio captured by the imaging unit 46 are stored in the storage unit 43 or output to the monitoring device 13 via the communication network 15 by the transmission / reception unit 50 in real time.

  The tracking / guidance means 47 detects a person included in the video obtained by the imaging means 46 by the above-described face area detection, human body area detection, etc., and is stored in the storage means 43 or obtained by the monitoring device 13. Check against specific person information such as suspicious person information and lost child information. Further, when the person included in the captured video is a suspicious person as a result of the collation, the tracking / guidance means 47 tracks the suspicious person. In addition, when the person included in the captured video is a lost child, a message is output so that the lost child is followed by voice or the like, and the lost child is guided to a predetermined position while being moved by the moving means 44. Process. The tracking / guidance means 47 collates the captured video with a specific person such as a suspicious person or a lost child continuously at a predetermined timing.

  This makes it possible to set a route in which the patrol is ended when the monitoring robot 14 finds the target person and the target person is guided to a predetermined place such as a management room.

  In addition, when the inter-mobile unit 48 uses a plurality of monitoring robots to circulate and monitor a certain point with priority, the location, time, contents, etc. that circulate between other monitoring robots. To control. For this reason, the inter-moving body cooperation means 48 cooperates between the monitoring robots 14 with each of the monitoring robots 14 while exchanging the current position and the route information between the monitoring robots through the communication network 15. You can monitor while taking it.

  As a result, it is possible to control a plurality of robots, such as always having one or more monitoring robots in the priority area, or using two robots that patrol the priority area at a time. it can.

  The notification means 49, when a specific person such as a suspicious person or a lost child is discovered during a warning signal (for example, fire, water leakage, person detection, etc.) or when a fire or water leak is detected, warning information to that effect. The monitoring device 13 is notified via the communication network 15. Further, for example, information such as “This is a prohibited area” is notified to the suspicious person. In addition, information such as “I'm okay” or “Please follow me” is notified to a lost child or the like.

  In addition, the notification means 49 notifies the monitoring device 13 of the current position periodically or at a predetermined timing such as movement between points according to a preset tour route. In addition, when the tour ends, the notification unit 49 notifies the monitoring device 13 of a tour end notification indicating that the tour has ended.

  For example, when notifying the monitoring robot 14 or the like, the notifying unit 49 is located so close that the monitoring robot 14 faces the subject or the subject is photographed by the imaging unit of the monitoring robot 14. Therefore, it is possible to output a voice message from the monitoring robot 14 to the subject, or to output a signal that causes a warning lamp, an emergency sound, or the like to call attention to the surroundings.

  The transmission / reception means 50 is transmitted from the imaging means provided in one or a plurality of monitoring cameras 11-1 to 11-n and monitoring robots 14-1 to 14-n via a communication network such as a LAN (Local Area Network) or the Internet. Receive surveillance video. Note that the transmission / reception means 50 does not have to receive the monitoring video directly from the monitoring camera 11. For example, the video acquired by the monitoring camera 11 is temporarily stored somewhere and the stored information is used. In this embodiment, human body detection may be performed.

  The transmission / reception means 50 can be used as a communication interface for transmitting to other terminals of the monitoring robot 14 and receiving various data from other terminals.

  The control means 51 controls the entire functional configuration of the monitoring robot 14. Specifically, the control unit 51 performs various controls such as executing the processing in each functional configuration described above based on the input information from the user input by the input unit 41.

  Further, for example, when the monitoring robot 14 is traveling around the monitoring area, the control unit 51 controls the monitoring robot 14 regarding the traveling route, the operation when the target person is found, and the like. Specifically, the control unit 51 causes the notification unit 49 to output a voice message, an alarm, or the like for the target person or its surroundings when the target person is found, or forcibly terminates the patrol of the monitoring robot 14 after the target person is found. Control such as setting a return path is performed. In other words, when the monitoring robot 14 finds the target person, the patrol can be ended and the route can be set so that the target person is guided to a predetermined place such as a management room.

  Furthermore, the control means 51 controls the monitoring robot 14 in response to a warning signal (for example, fire, water leakage, person detection, etc.) obtained from the monitoring robot 14 such as an emergency operation or notification to the monitoring device 13. To do.

  By using each device configuration described above, it is possible to perform a patrol with an appropriate movement route using the feature search function.

  Specifically, in the past, the feature search device and the monitoring robot have been independent systems, and the cooperation between the systems only requires a threatening voice output request, etc. There wasn't.

  However, by applying the present embodiment, it is possible to designate a traveling point for the monitoring robot based on the search result in the feature search function.

<Search Process in Search Unit 30>
Here, as a search process in the present embodiment, an outline until the search person feature is acquired will be described. FIG. 4 is a diagram for explaining an outline until a search person feature is acquired in the present embodiment. The diagram shown in FIG. 4 is a diagram for generating search person feature data from the person features obtained by the above-described face region detection and human body region detection. The person characteristic data is generated by the frame information generating unit 29 and the like, and is stored in the storage unit 23 by a database or the like.

  Specifically, in FIG. 4A, in frame information generation, characteristics (for example, height, presence / absence of sunglasses, etc.) for the person (for example, person A, person B, etc.) detected in each frame (F). Is acquired, and these person features are collected for each person as shown in FIG. 4B, and the person features are counted. That is, in FIG. 4B, all data are discretized and the number of appearances is counted. Specifically, in FIG. 4B, the person A is extracted in three frames in the video, of which When the height is estimated to be 170 cm, there are 2 times, when the height is estimated to be 175 cm, once when the wearing of sunglasses is detected, twice, when there is no detection of wearing, etc. Is counted. In the above example, the height is discretized every 5 cm, but the present invention is not limited to this.

  Next, as shown in FIG. 4C, normalization is performed with the number of frames, and this data is used for retrieval. Specifically, when the search person feature data is created, in order to search for a person with a combination of various features and actions, for example, it is output in various forms such as height, sex, and HSV value of clothes color. According to the following formula, describe the person information in a unified manner.

For example, when creating data about a feature X of a person A, when the feature X is a digital value (gender (male / female), sunglasses (presence / absence), mask (presence / absence), etc.), The total frequency F _AX for the person A having the frequency value X is calculated by the equation (1) shown below.

また、特徴Ｘがアナログ値（身長、認証スコア、服の色のＨＳＶ値等）の際には、以下に示す（２）式により連続値Ｘ内特定範囲ｄの人物Ａについての総頻度を算出する。

When the feature X is an analog value (height, authentication score, clothes color HSV value, etc.), the total frequency of the person A in the specific range d within the continuous value X is calculated by the following equation (2). To do.

これにより、人物の全ての情報をヒストグラムへと変換することができる。なお、このままでは人物間の比較に用いることができないため、各人物の検出フレーム数で正規化することで、正規化ヒストグラムへと変換し、以下に示す（３）式により人物Ａの特徴Ｘについての信頼度Ｐ_ＡＸを算出する。

Thereby, all the information of the person can be converted into a histogram. Since it cannot be used for comparison between persons as it is, it is converted into a normalized histogram by normalizing with the number of detected frames of each person, and the characteristic X of the person A is expressed by the following equation (3). The reliability P _AX is calculated.

これにより、図４（ｃ）に示すように検索用人物特徴データを取得することができる。

Thereby, as shown in FIG.4 (c), the person characteristic data for a search can be acquired.

<Search method in this embodiment>
Next, the search method in the present invention in the search means 30 described above will be described with reference to the drawings. In the search method according to the present embodiment, it is possible to search for a person shown in a video with a combination of various person characteristics and actions by the above-described search scoring method using person feature data. In any combination, search results can be output in descending order of matching with the search request.

  Here, FIG. 5 is a diagram illustrating an example of a search method according to the present embodiment. In this embodiment, as shown in FIG. 5A, the search request “175 cm person wearing sunglasses” input by the searcher is converted into the above-described feature amount, and the maximum value ( = 1) is set as the search request feature amount (FIG. 5B). Further, a feature corresponding to the search request feature value is extracted with reference to the feature (FIG. 5C), a distance from the feature is calculated, and the distance is set as a search score (FIG. 5D). The search results are output in descending order of the number of persons.

  It should be noted that the color information suitable for search in the present embodiment is particularly important for the human characteristics, such as the color of clothes and hair, but the expression of the color changes depending on the subjectivity. To absorb. Specifically, in order to adapt the search result to a color that a person feels, for example, processing is performed in a color space that is set in advance based on human visual characteristics, such as an HSV space. The color space described above is not limited to HSV in the present invention, and for example, HLS, LUV, LAB, or the like can be used. Furthermore, the subjectivity variation is absorbed by reducing the color resolution.

  Further, when performing a search using free words, the input word / phrase is converted into a search request and a search process is performed. Conventionally, when a search keyword such as “red jacket blue pants” is entered, there is a problem that it is impossible to understand which adjectives and which nouns are in a modification relationship. Therefore, in the present embodiment, the search accuracy can be further improved by converting into a search request in consideration of the modification relationship of the input words. In this embodiment, when inputting a search request, it is possible to select from a free word input representing a person or a searchable person feature list.

  Thereby, it is possible to easily search for a video at a time when a person close to a person image intended by the searcher is taken from a huge amount of videos stored, and to obtain the search result. In the present embodiment, when the video is confirmed, in addition to the person video, the appearance characteristics and behavior of the person are written in characters. The desired person can be output to various media as suspicious person information. Thereby, a highly accurate person search can be efficiently performed with a simple operation on the captured image.

  Next, the operation flow using this embodiment will be specifically described. In the example shown below, a flow relating to (1) real-time search, (2) suspicious part patrol, and (3) processing after extraction of a designated characteristic person will be described in order. In the following processing, a sequence flow using the monitoring device 13 and the monitoring robot 14 will be described.

<(1) Real-time search>
FIG. 6 is a sequence chart showing an example of an operation processing procedure during real-time search in the present embodiment. In the operation processing procedure shown in FIG. 6, when there is a real-time search request from the user or the like in the monitoring device 13 (S01), the search in the present embodiment is performed as described above from the image stored in advance (S02), A score for each place is calculated (S03). Moreover, a candidate point is selected from the result (S04), a patrol request is transmitted to the monitoring robot 14 based on the selected result (S05), and a real-time search is started (S06).

  Further, when the patrol request information is acquired from the monitoring device 13 (S07), the monitoring robot 14 determines whether the patrol is currently in progress (S08). Here, when the tour is in progress (YES in S08), the tour is stopped (S09), and the tour route from the current position is reset (S10). Further, in the process of S08, if the circuit is not in circulation (NO in S08), a circulation route is set based on the circulation information set in advance and the circulation request information received in S07 (S11), and the process of S10 or S11 is performed. After completion, patrol is started (S12).

  That is, in the real-time search described above, a feature search is performed from the stored image based on the specified feature, and a specific person obtained as a result of the search is classified by location from the identification information of the monitoring camera 11 that captured the image. Calculate the score for each location. Further, based on the calculated score, a priority patrol point is selected, and a patrol request is transmitted to the monitoring robot 14 or the like responsible for patrols at or near that point. In addition, when the surveillance robot 14 receives the patrol request, the surveillance robot 14 calculates the patrol route so that the patrol point in the patrol request information can be patroled, and performs patrol.

<Course request information>
Here, a specific example of the patrol request information will be described. The tour request information in the present embodiment includes, for example, data such as an information type indicating the type of the request information and priority tour point information.

  In the information type, for example, a type such as real-time search, tracking, and suspicious alert route is set. In the above example, “real-time search” is set as the information type. In the priority patrol point information, for example, (Point A, 100.0), (Point B, 90.0),. Here, X (Point A etc.) of the above-mentioned priority patrol point information (X, Y) indicates the coordinates of the point, and Y (100.0 etc.) indicates the priority of the tour.

  In this way, by setting the priority patrol point information, it is possible to easily set the patrol route. Further, based on the information identification included in the patrol request information, the monitoring time and monitoring contents of the monitoring robot are set. Note that the information included in the patrol request information is not limited to this in the present invention, and may include, for example, search target person information (face, hair, clothing characteristics, etc.).

<(2) Suspicious part patrol>
Next, suspicious part patrol in this embodiment will be described. FIG. 7 is a sequence chart illustrating an example of an operation processing procedure when visiting a suspicious part. In FIG. 7, the monitoring device 13 first manages the robot state of the monitoring robot 14 (S21), and determines whether or not it is currently patroling (S22). If the circuit is in circulation (YES in S22), the process returns to S21 to perform subsequent processing. If the circuit is not in circulation (NO in S22), a suspicious search is performed from images stored in the storage unit 23 in advance (S23), and a location-specific score is calculated based on the suspicious search result (S24). In the process of S23, when it is determined that a suspicious person is present in the image, a plurality of information is obtained using the identification information of the monitoring camera 11 that captured the image, the shooting time, the similarity between the suspicious person, and the like. Based on the suspicious person search result of each image obtained from the monitoring cameras 11-1 to 11-n, the score is calculated for each location by the process of S24.

  Next, the suspicious degree of the suspicious person obtained by the process of S23 is compared with a preset threshold value. Specifically, it is determined whether or not the suspicious degree is larger than the threshold value (S25). If the suspicious degree is not larger than the threshold value (NO in S25), the process returns to S23 to perform subsequent processing. In the process of S25, when the suspicious degree is larger than the threshold value (YES in S25), a patrol request is transmitted to the monitoring robot 14 (S26).

  When the surveillance robot 14 receives a patrol request (S27), the surveillance robot 14 sets a patrol route (S28) and starts a guard patrol (S29). Further, while patroling the predetermined route (S30), it is determined whether the patrol is finished (S31). If the patrol is not finished (NO in S31), the process returns to S30 until the patrol is finished. Continue processing.

  When the tour is completed (YES in S31), the monitoring device 13 is notified of a tour end notification indicating that the tour is completed (S33). When the monitoring device 13 receives the patrol end notification from the monitoring robot 14, the monitoring device 13 ends the processing (S34).

  In other words, in the above-described processing, the monitoring device 13 that performs feature search manages the state of the monitoring robot 14 and performs suspicious searches at regular intervals when the robot is not patroling. In addition, the monitoring device 13 classifies the search results for each place, and when there is a place where the score exceeds a certain threshold, the place is selected as a priority patrol point, and a patrol request is transmitted to the monitoring robot. When receiving the tour request, the monitoring system 14 calculates a tour route based on the priority tour point information in the tour request information, and starts the tour. In the processing of S26 and S27 described above, the traveling request information transmitted and received is the same as the above-described content, but information identification is suspicious.

<(3) Processing after a specified feature person is extracted>
Next, a process after the designated feature person is extracted in the present embodiment will be described. FIG. 8 is a sequence chart showing an example of an operation processing procedure at the time of processing after the designated feature person is extracted. In FIG. 8, first, when the monitoring device 13 receives an input of a designated feature (S41), the monitoring device 13 performs a search from the stored image (S42).

  Next, the person with the highest score is extracted from the search result (S43), and the destination is set based on the extracted information (S44). In the process of S44, the location where the surveillance camera 11 is installed or the location where the surveillance camera 11 is installed and its image are taken based on the surveillance camera 11 or the like that has acquired the image where the person with the highest score was taken. The destination is set on the basis of the elapsed time from and the direction of movement of the person viewed with time.

  Here, it is determined whether or not the destination set in the process of S44 has changed from the preset destination (S45). If the destination has changed (YES in S45), the destination The patrol request information for the destination is transmitted from the monitoring device 13 to the monitoring robot 14 (S46). In the process of S46, personal information to be searched (face, hair, clothing characteristics, etc.) can also be transmitted.

  Upon receiving the patrol request information from the monitoring device 13 (S47), the monitoring robot 14 sets a patrol route (S48), and starts a guard patrol based on the set patrol route (S49). Further, patrol is performed (S50), and it is determined whether or not a target person (suspicious person or the like) has been found (S51). Here, when the target person has not been found (NO in S51), the process returns to S50 and the circulation is continued. If the target person is found (YES in S51), a notification of discovery is sent to the monitoring device 13 (S52). Thereafter, tracking is performed (S53), and it is determined whether or not it is lost (lost) (S54). Here, when it is not lost (in S54, NO), it returns to S53 and continues tracking. Further, if the user loses sight (YES in S55), a lost notification is sent to the monitoring device 13 (S55).

  Here, the monitoring device 13 receives the notification result from the monitoring robot 14 (S56), and determines whether or not a target person has been found (S57). If it is found (YES in S57), the process returns to S56 and continues to wait for notification of the tracking result. If not found (NO in S57), the process returns to S42 to perform subsequent processing from the designated feature person search.

  That is, in the above-described example, when the designated feature is input, the monitoring device 13 extracts the person whose highest score is calculated from the accumulated image. Further, the place where the person is confirmed is set as the destination, and a patrol request is transmitted to the monitoring robot.

  Upon receiving the tour request, the monitoring robot 14 calculates a tour route so as to reach the destination in the tour request information, and starts the tour. If the designated person is included in the image captured by the imaging means provided in the monitoring robot 14, a discovery notification is transmitted to the monitoring device 13, and tracking is further started. In addition, when the target person is lost during tracking, a lost notification is transmitted to the monitoring device 13. When the monitoring device 13 receives the discovery notification, the monitoring device 13 temporarily stops the patrol request to the monitoring robot 14, and when the lost notification is received, the search from the stored image is started again. The above-described operation is continued until an end notification is given from the user.

  According to the above-described embodiment, the surveillance robot 14 that is an autonomous mobile body sets up a tour route at random, cooperates with the score of the feature search, and effectively uses the score to efficiently operate the surveillance robot. Is possible. In addition, crime can be prevented by the monitoring robot while reducing the burden on the person on the monitoring system. In the above-described embodiment, only one monitoring robot 14 is controlled. However, the present invention is not limited to this, and the same monitoring device 13 can be used for a plurality of monitoring robots. Alternatively, different control can be performed depending on the patrol position or the like for each monitoring robot.

<Other embodiments>
In the above-described embodiment, the tour route is set on the monitoring robot side. However, the present invention is not limited to this, and the tour route is set for each monitoring robot 14 on the monitoring device 13 side. be able to. In this case, the monitoring device 13 is provided with the above-described traveling route setting means. Here, the above-described content will be described as another embodiment.

<Other embodiments: patrol route setting on the monitoring device side>
FIG. 9 is a sequence chart illustrating an example of an operation processing procedure when setting a patrol route on the monitoring device side. In FIG. 9, first, the monitoring device 13 performs a search from the stored image (S61), and calculates a location-specific score from the search result (S62).

  Next, it is determined whether or not the suspicious degree is larger than a preset threshold value (S63). If the suspicious degree is not greater than the threshold value (NO in S63), the process returns to S61 to perform subsequent processing. If the suspicious degree is larger than the threshold value in S63 (YES in S63), the robot position information is requested to the monitoring robot 14 (S64).

  The monitoring robot 14 receives the position information request from the monitoring device 13 (S65), and transmits the current position information to the monitoring device 13 (S66). At this time, identification information and time information of the monitoring robot 14 may be included simultaneously with the current position.

  Next, the monitoring system 13 receives the robot position information from the monitoring robot 14 (S67), and sets a patrol route (S68). Further, the set tour request information is transmitted to the monitoring robot 14 (S69). The monitoring robot 14 receives the patrol request information from the monitoring device 13 (S70), determines whether or not the patrol is currently in progress (S71), and stops patrol if the patrol is in progress (YES in S71) (S71). S72). Moreover, when it is not in patrol (in S71, NO), or after the process of S72 is complete | finished, a guard patrol is started (S73) and a patrol is performed (S74).

  In other words, in the above-described embodiment, when the patrol condition is satisfied in the suspicious search result, the monitoring robot 14 is requested for position information. When the current position of the robot is received from the monitoring robot 14, a tour route is calculated based on the suspicious place information, and a tour request is transmitted to the monitor robot 14.

  The tour request information in the present embodiment described above is configured to include an information type and route information. In the above example, “suspicious alert route” is set as the information type. In the route information, for example, (Point A, work 1, work 2,...), (Point O, work 1, work 2,...),. Here, X (Point A etc.) of the above-mentioned priority patrol point information (X, Y1, Y2,...) Indicates the coordinates of the point, and Y1 (work 1 etc.), Y2 (work 2 etc.) are the robots at the coordinates. Operation (work) contents (for example, operations such as camera control provided in the monitoring robot 14) are shown.

  In this way, by setting the priority patrol point information, it is possible to easily set the patrol route. Further, based on the information identification included in the patrol request information, the monitoring time and monitoring contents of the monitoring robot are set. Note that the information included in the patrol request information is not limited to this in the present invention, and may include, for example, search target person information (face, hair, clothing characteristics, etc.). In the example described above, a plurality of monitoring robots can be operated.

<Operation flow when the monitoring robot has a search function>
Furthermore, in the present invention, a feature person search function in the monitoring device 13 described above from the monitoring robot 14 may be provided. In this case, the monitoring robot 14 is provided with the search means described above. Here, specific examples of the above contents will be described below.

  FIG. 10 is a sequence chart illustrating an example of an operation processing procedure when the search function is provided in the monitoring robot. Note that the processing of FIG. 10 can be set without using the monitoring device 13, so an example using the monitoring camera 11 and the monitoring robot 14 will be described with reference to the drawings. The present invention is not limited, and the entire processing management may be performed using the monitoring device 13.

  In FIG. 10, first, images obtained from the respective monitoring cameras 11-1 to 11-n are transmitted to the monitoring robot 14 (S81). The surveillance robot 14 receives images taken from the surveillance cameras 11-1 to 11-n (S82) and accumulates the received images (S83). At this time, images taken from imaging means such as a camera provided in the monitoring robot 14 may be accumulated.

  Next, the surveillance robot 14 performs a search from the stored image (S84), and calculates a location-specific score from the search result (S85).

  Next, it is determined whether or not the suspicious degree is larger than a preset threshold value (S86). If the suspicious degree is not greater than the threshold value (NO in S86), the process returns to S84 to perform subsequent processing. If the suspicious degree is larger than the threshold value in S86 (YES in S86), a patrol route is set (S87). Further, it is determined whether or not the tour is currently being performed (S88). If the tour is being performed (YES in S88), the tour is stopped (S89). Moreover, when it is not in patrol (in S88, NO), or after the process of S89 is completed, the patrol patrol is started (S90), and patrol is performed (S91).

  In the above-described process according to the present embodiment, the location-specific score may be calculated based on the sensor detection information from each sensor group 12. In this case, since no image is included in the sensor, detection position information and detection identification information are transmitted to the management robot 14.

<Example of Screen Generated by Screen Generation Unit 33>
Next, a screen example generated by the screen generation unit 33 will be described with reference to the drawings. FIG. 11 is a diagram illustrating an example of screen transition in the present embodiment. The screen example shown in FIG. 11 includes an initial screen 61, a date / time designation screen 62, a feature / action condition designation screen 63, a search result display screen 64, a video playback screen 65, and a shared screen 66. ing. When the system according to the present embodiment is activated, an initial screen 61 is first displayed. Further, the transition to each screen shown in FIG. 11 is performed by setting a transition button for transitioning to a predetermined screen as shown in FIG. 11 and selecting the transition button so that each screen is output means such as a display. 22 is output. In addition, the example of the screen produced | generated in this invention is not limited to this, Moreover, the screen layout and display content shown below are not limited to this.

  There are five search methods from the initial screen 61, for example, store visitor search, suspicious person search, keyword search, condition-designated search (person feature search), and real-time search.

  For example, in the target facility in which the above-described surveillance camera 11 is installed (for example, when confirming a person who has visited a store, the display switches to the date and time designation screen 62 by pressing a “visitor search button” for searching for a store visitor. When searching for a suspicious person on the initial screen 61, the date and time designation screen 62 is displayed by pressing a suspicious person search button.

  In addition, in the initial screen 61, when performing a keyword search, a keyword to be searched is input in the keyword search edit box, and a “keyword search” button is pressed, so that date and time information set in advance, etc. When the search means 30 performs a search using the search characters entered in the edit box as keywords (search request conversion processing), the screen transitions to the search result display screen 64 and the search results are displayed. Is displayed.

  Further, in the initial screen 61, when searching from the feature of a person, the “Transition from person feature” button is pressed to transition to the feature / action condition designation screen 63.

  Furthermore, in the initial screen, when performing a real-time search for a lost child or the like by the monitoring robot, the “real-time search button” is pressed to transit to the feature / action condition designation screen 63.

  In the date and time designation screen 62, date and time conditions and search location conditions are designated. Thus, for example, even when an image obtained by a moving imaging means such as a monitoring robot is targeted, an image obtained by a target location based on the patrol position information of the monitoring robot obtained together with the image. Can be efficiently searched for only the target. This location designation is effective because, for example, the video to be searched can be limited even when the video from the fixed imaging means is targeted.

  Next, the designated date and time conditions and place conditions are passed to the search process, and when the search process is completed, the search result is called and the screen shifts to the search result display screen 64. The designated place may be, for example, a place name, a postal code, a name of a store or a commercial facility, a floor number of a floor, and the like, and is further designated as an area range within a radius of 100 m from the monitoring robot that captured the video. You can also.

  The feature / behavioral condition designation screen 63 designates a search condition when “search from person feature” or “real-time search” is selected. Call the storage function for all conditions. Further, when the search button is pressed, a search processing function is called and a transition is made to the search result display screen 64.

  Also, on the feature / action condition designation screen 63, each designation such as date designation, place designation, height designation, color information designation, gender designation, mask designation, sunglasses designation, age designation, character designation, similar entertainer designation, etc. An area, a search execution button group, and a hyperlink display area.

  In the place designation described above, a place for searching for a target person is set. Specifically, the location information to be searched is specified based on the location information of the surveillance camera 11 where the video is taken. Further, if the position information is a fixed type such as the monitoring camera 11, it can be easily obtained from the identification information of the imaging means included in the video data, and if the position information is a moving type such as a monitoring robot. The patrol position information of the monitoring robot included in the video data can be used.

  In the character designation, a registered person name is set from the list in accordance with registered person information accumulated in advance. Also, in the case of similar entertainer designation, entertainer names are set from the list according to entertainer information accumulated in advance.

  The search execution button group includes a “search person for this condition” button and a “clear button”. When the “search person for this condition button” is pressed, the “person” specified on the initial screen 61 is displayed. The storage function for all conditions corresponding to “search from feature” or “real-time search” is sequentially called. Finally, the search function is called to execute the search process, and after the process is completed, the screen transitions to the search result display screen 64. If there is an input that does not meet the conditions, for example, a message “Please input again” is notified. When the “clear button” of the search execution button group is pressed, the settings of various conditions on the feature / action condition designation screen 63 are cleared (initialized). Further, in the hyperlink display area 68, by selecting a portion where characters are displayed, the screen is changed to the first screen (initial screen 61).

  In the present embodiment, various search conditions are specified on the initial screen 61, the date / time specification screen 62, and the feature / action condition specification screen 63 described above, and the search results are displayed on the search result display screen 64. Also, a predetermined search result output function is called to display various information.

  The search result display screen 64 has various search areas, a search result number display area, an image display area when a corresponding person appears, a face image of the corresponding person, a search result display area, and the like. It has an icon and a hyperlink display area.

  In various search areas, according to the event selected on the initial screen 61 or the like, the search process and screen transition are performed by pressing various buttons corresponding to the event. Each search area is provided with a “pause button”. This button is used to temporarily stop the output of the real-time search result without updating the real-time search, for example. is there. Specifically, for example, in the case of real-time search, the search result display area is displayed as needed from the one with high similarity to the person to be searched such as a lost child, or the result having a certain similarity or more Is displayed at any time, the display content is updated, and there is a risk of overlooking the content displayed on the screen. Therefore, the search result can be confirmed accurately by pausing the update display of the search result.

  In the search result number display area, the number of search results and the number of currently displayed results are displayed (for example, “1-5 of 62 search results”). Further, in the image display area when the corresponding person appears, the human body image for thumbnail when the corresponding person appears is displayed. In addition, in the face image of the corresponding person, a thumbnail face image of the corresponding person is displayed. In the image display area when the person appears and the face image of the person, the image is not displayed if there is no display target image (“NO IMAGE” display or the like).

  In addition, the search result display area displays icons corresponding to the information of the corresponding person (appearance features / behavior) or detection by the monitoring robot as a result of the search.

  The icons described above are set in advance corresponding to the above-described conditions, and can be displayed separately in preset colors such as “red”, “black”, and “gray”. Here, if it is displayed in “red”, it indicates that it was included in the search condition, and “black” indicates that it is not included in the search condition but is an image that meets the above condition. “Gray” indicates that the search condition and the analysis result of the video are not applicable. Note that the type of color is not particularly limited in the present invention, and other colors may be used. Alternatively, the colors may be distinguished and highlighted by blinking, diagonal lines, or the like. In the icon display area, it is also possible to display the character itself instead of the icon. In this case, for example, the font type, size, thickness, underline, etc. of the character can be changed or blinked. It can be distinguished by changing the color.

  Further, in the video confirmation button, when the button is clicked, a video playback process is called and a transition to the video playback screen 65 is made. Further, the page selection icon jumps to each page when the search result covers a plurality of pages. Further, in the hyperlink display area 76, the screen is changed to the first screen (initial screen 61) by selecting the portion where the character is displayed. The above-described screen allows the searcher to easily grasp which condition the extracted search result corresponds to.

  That is, on the search result display screen 64, by displaying information such as a person's characteristics and behavior, devices linked to the imaging means 26, and the like with icons, the requested person can be easily confirmed.

  The video playback screen 65 has a video display area, a text display area, a feature display area, and a button area. In the video display area, video is played back and displayed using the input file path, playback start time, and playback end time as arguments of video playback processing. In addition, the video display area includes a slider, a playback button, a stop button, and the like, so that the user can easily view the video from a place where he / she wants to view the video, and play / stop the video. In addition to the AVI file, the data displayed in the video display area includes MPEG2, MPEG4, H.264. H.264, WMV, MP4, and other file format data can be displayed, and image data such as JPEG, GIF, TIFF, and BMP can also be displayed.

  In the text display area, when the video playback time is the time in the text, the action text corresponding to the video can be displayed with highlighted characters such as red letters and bold letters. Further, past information is displayed as it is, and when new information is displayed, the past information is changed to black characters. Thereby, it is possible to easily grasp the current action and the action history so far.

  In the feature display area, the feature information of the person included in the displayed video is displayed. When a plurality of people are photographed, the respective feature information is displayed.

  In the button area, a “share button” and a “similar search button” are provided. When the “share button” is pressed, the screen transitions to the share screen 66. In this case, the video reproduction in the video display area is stopped to reduce the load on the apparatus. Also, when the “similarity search button” is pressed, a similarity search from another video is performed based on the characteristics of the displayed person, etc., and it is checked whether the image is taken by another camera, and the result is the search described above. Output after transitioning to the result display screen 64.

  That is, in the video playback screen 65, since the action and time of the person in the video are described in the text passed from the search result display screen 64, the video is displayed while the video is being displayed in the video display area. When the time corresponding to the text is reached, the text is displayed in red in the text display area (highlighted). Thereafter, when there is new information, the past information is displayed in black characters.

  In addition, when the “share button” in the button area is pressed on the video playback screen 65 described above, the person information is displayed on the share screen 66. Here, the shared screen 66 has an image display area, a text display area, and a comment input area. In the image display area, an image when the person appears and a face image of the person are displayed. In the text display area, suspicious information is displayed in highlighted characters such as red letters, and time information (date, time of entry, time of exit, etc.) and characteristics (height, gender, year, etc.) are also displayed. The In the comment input area, a comment is input by the input means 21. The comment input area displays “Please input a comment” as a default display, and the input means 21 can accept input of characters by a searcher or the like. Thereby, the searcher can set and manage personal information from a unique point of view.

  Furthermore, it has a print / output function for outputting various contents displayed on the shared screen 66 to a printer, generating a file, and sending it by e-mail to a predetermined mobile phone. This makes it possible to check information about a person from both text and video on the video display screen, and to output it to an external device in the form of printed matter, etc. (Sharing).

<Robot route setting screen>
Next, an example of the robot path setting screen in the above-described embodiment will be described. FIG. 12 is a diagram illustrating a screen example of robot path setting in the present embodiment. The robot route setting screen 71 shown in FIG. 12 is configured to have a route map display region 72, a route candidate selection region 73 for each robot, a tour route information display region 74, and a route edit region 75.

  In the route map display area 72, the selected tour route is displayed. The route map display area 72 can also display each point set in advance, a place where the monitoring camera 11 or each sensor group 12 is installed, or the like.

  Furthermore, the route map display area 72 can also display the monitoring video taken by the monitoring robot and the patrol position information (three-dimensional coordinates, latitude, longitude, etc.) transmitted together with the monitoring video capital.

  The robot-specific route candidate selection area 73 displays a list of route candidates calculated for each robot. Thereby, the user can grasp | ascertain a route candidate for every monitoring robot easily. The traveling route information display area 74 displays a list of passing coordinates of the selected route. Furthermore, the route editing area 75 performs editing of the selected route, editing of the robot operation for each point, and the like.

  In the example of FIG. 12, the route calculation of the plurality of monitoring robots 14-1 to 14-n can be performed on the monitoring device 13 side. In the example of FIG. 12, an example of a screen at the time of calculating a robot-specific tour route is shown.

  That is, on the robot route setting screen 71, route candidates for each monitoring robot and monitoring robot operations for each point are displayed and can be edited by the user. If selection or editing by the user is not performed, the highest route candidate is selected.

<Execution program>
Here, the above-described monitoring system using the autonomous mobile body (such as the monitoring device 13 and the monitoring robot 14) includes, for example, a CPU (Central Processing Unit), a volatile storage medium such as a RAM (Random Access Memory), and a ROM (Read). By incorporating a non-volatile storage medium such as (Only Memory), a touch panel, a mouse, a keyboard, an input device such as a pointing device, a display device for displaying images and data, and a computer equipped with an interface device for communicating with the outside Can be configured.

  Accordingly, the above-described functions of the monitoring device 13 and the monitoring robot 14 can be realized by causing the CPU to execute a program describing these functions. These programs can also be stored and distributed in a recording medium such as a magnetic disk (floppy disk, hard disk, etc.), optical disk (CD-ROM, DVD, etc.), semiconductor memory, or the like.

  In other words, it is possible to realize the monitoring process by generating an execution program (monitoring program) for causing the computer to execute the processing in each configuration described above and installing the program in, for example, a general-purpose personal computer or server. it can. In addition, about the process by the execution program in this invention, each process mentioned above is realizable, for example.

  As described above, according to the present invention, a monitoring system, a monitoring device, an autonomous moving body, a monitoring method, and a monitoring program using an autonomous moving body for performing a tour along an appropriate movement route using a feature search function are provided. Can be provided.

  As for the person search, there are a case where a person is searched for a video stored in advance, or a case where a person is searched for a video obtained in real time. There is a fixed type installed on a set wall or ceiling, etc., and a mobile type mounted on a surveillance robot, etc. According to the present invention, an appropriate person search corresponding to any of them is realized. can do.

  Further, according to the present invention, for example, by the search scoring method described above, it is possible to search for a person shown in an accumulated video or a real-time video by a combination of various person characteristics and actions. In any combination, search results are output in descending order of matching with the search request. As a result, the search result can be output effectively, and the work of checking the video can be greatly reduced. Furthermore, according to the present invention, it is possible to perform a similarity search of persons between videos and display the result.

  In other words, the surveillance robot can be efficiently operated by cooperating with the score of the feature search and effectively utilizing the score, rather than the surveillance robot randomly setting the patrol route. The crime can be prevented by the surveillance robot while reducing the burden on the surveillance system.

  The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to such specific embodiments, and various modifications, within the scope of the gist of the present invention described in the claims, It can be changed.

DESCRIPTION OF SYMBOLS 10 Movement management system 11 Monitoring camera 12 Each sensor group 13 Monitoring apparatus 13 Monitoring robot 15 Communication network 21, 41 Input means 22, 42 Output means 23, 43 Accumulation means 24 Face / human body area detection means 25 Identification means 26 Tracking means 27 Suspicious means 27 Person detection means 28 Person information integration means 29 Frame information generation means 30 Search means 31 Important monitoring point selection means 32 Travel instruction means 33 Screen generation means 34, 49 Notification means 35, 50 Transmission / reception means 36, 51 Control means 44 Movement means 45 Travel Route setting means 46 Imaging means 47 Tracking / guidance means 48 Inter-mobile body cooperation means 61 Initial screen 62 Date and time designation screen 63 Feature / action condition designation screen 64 Search result display screen 65 Video playback screen 66 Shared screen 71 Robot path setting screen 72 Route Map display area 73 Tsu door by the route candidate selection area 74 patrol route information display area 75 route editing area

Claims (15)

In a monitoring system comprising at least one autonomous moving body that circulates a monitoring area on a predetermined route, and a monitoring device that monitors the monitoring area using the autonomous moving object,
The monitoring device
Specific person extraction means for extracting a specific person using an image taken by at least one imaging means installed in the monitoring area;
An important monitoring point selecting means for selecting an important monitoring point from position information of an image pickup means that has taken an image obtained by the specific person extracting means;
Generating patrol request information corresponding to the point obtained by the important monitoring spot selecting means, and having the patrol instruction means for instructing the generated autonomous mobile body with the patrol request information generated,
The autonomous mobile body is
A monitoring system comprising: a tour route setting means for setting a tour route using the tour request information obtained from the monitoring device and a preset tour route. The important monitoring point selecting means is:
The monitoring system according to claim 1, wherein a score is calculated for each of a plurality of places set in advance in the monitoring area. The patrol request information is:
The monitoring system according to claim 1, comprising the important monitoring point and a priority for each important monitoring point. The patrol route setting means includes:
When the autonomous mobile body is traveling, the current position information is acquired, and the traveling route is set based on the distance between the acquired position information and the important monitoring point. 4. The monitoring system according to any one of 3. In a monitoring device that monitors the monitoring area using at least one autonomous mobile body that patrols the monitoring area on a predetermined route,
Specific person extraction means for extracting a specific person using an image taken by at least one imaging means installed in the monitoring area;
An important monitoring point selecting means for selecting an important monitoring point from position information of an image pickup means that has taken an image obtained by the specific person extracting means;
A monitoring apparatus comprising: tour request information that generates tour request information corresponding to a point obtained by the important monitor spot selecting unit, and that instructs the generated travel request information to the autonomous mobile body. The important monitoring point selecting means is:
The monitoring apparatus according to claim 5, wherein a score is calculated for each of a plurality of places set in advance in the monitoring area. The patrol request information is:
The monitoring apparatus according to claim 5, comprising the important monitoring point and a priority for each important monitoring point. In an autonomous moving body that patrols a monitoring area on a predetermined route according to an instruction from a monitoring device,
Using a patrol request information obtained from the monitoring device and a patrol route set in advance, a patrol route setting means for setting a patrol route;
Moving means for moving based on the cyclic route obtained by the cyclic route setting means;
An autonomous mobile body comprising imaging means for capturing an image of surroundings moving along the patrol route. The patrol route setting means includes:
9. When the autonomous mobile body is traveling, the current position information is acquired, and the traveling route is set based on the distance between the acquired position information and the important monitoring point. The autonomous mobile body described. In a monitoring method in a monitoring system comprising at least one autonomous mobile body that circulates a monitoring area on a predetermined route, and a monitoring device that monitors the monitoring area using the autonomous mobile body,
A specific person extracting step of extracting a specific person by using the image taken by at least one imaging means installed in the monitoring area by the monitoring device;
An important monitoring point selection step of selecting an important monitoring point from the positional information of the imaging means that took the image obtained by the specific person extraction step;
Generating a tour request information corresponding to the point obtained by the important monitoring spot selection step, and a tour instruction step for instructing the generated travel request information to the autonomous mobile body;
A monitoring method comprising: a patrol route setting step for setting a patrol route by using the patrol request information obtained from the monitoring device and a preset patrol route by the autonomous mobile body. The important monitoring point selection step includes:
The monitoring method according to claim 10, wherein a score is calculated for each of a plurality of places set in advance in the monitoring area. The patrol request information is:
The monitoring method according to claim 10 or 11, comprising the important monitoring point and a priority for each important monitoring point. The patrol route setting step includes:
When the autonomous mobile body is traveling, the current position information is acquired, and the traveling route is set based on the distance between the acquired position information and the important monitoring point. 13. The monitoring method according to any one of items 12.   A monitoring program for causing a computer to function as the monitoring device according to any one of claims 5 to 7.   A monitoring program for causing a computer to function as the autonomous mobile body according to claim 8 or 9.

Images