JP2006229465A - Monitoring apparatus, monitoring method, and monitoring program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To simply and surely identify and mark-display a marked person without increasing the processing load of a recognition processing. <P>SOLUTION: In the monitoring apparatus, a video processing section 2 processes video images from a plurality of monitoring cameras 1 and displays the processed image on a display section 8. In this case, a marking section 4 applies marking to a person displayed on the display section 8 according to operations by an operation section 3, and when a marking information storage section 5 stores a feature quantity of the marked person, a person identification section 6 applies person identification discrimination to the person in the video images from the monitoring cameras 1 and the person in the marking information storage section 5 through a plurality of image recognition processes. In this case, a recognition processing limit section 7 applies switching control of the image recognition processes to the video images from a plurality of the cameras 1 to control avoidance of the recognition processing applied to the other camera video images when the mark display of the marked person is made, thereby suppressing the load of the image recognition processes. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a monitoring apparatus, a monitoring method, and a monitoring program for monitoring a monitoring target such as a person or the like marked by a monitoring person using an imaging device such as a plurality of monitoring cameras.

  With the recent increase in the number of crimes, video surveillance systems that monitor suspicious persons using video from surveillance cameras are becoming popular. In addition, surveillance cameras have been converted to IP, and more and more surveillance cameras have been monitored over a wide area. In such a situation, when monitoring is performed by paying attention to a specific person, the supervisor must track a person moving between multiple cameras while switching the cameras, which requires a great deal of labor. It was. Therefore, there is a need for a technique for reliably and efficiently performing person monitoring work.

As a conventional monitoring device, an image of a person's face projected on a monitor is marked by a supervisor, and a specific person is discriminated from another person by comparing the camera image with the marked person's face image. There are some which are displayed in the form (for example, see Patent Document 1).
JP 2004-128615 A (FIG. 2)

  However, in the above-described conventional monitoring device, if a false detection occurs in face matching due to conditions such as the tilt and angle of the face, marking is performed when multiple cameras are arranged at intervals that do not share the field of view. The person should be taken by only one camera at a time, but another person in multiple camera images will be recognized as a marked person, and several other persons will be marked at the same time There is a problem of being.

  On the other hand, in addition to face matching, a method for improving the accuracy of human mark display by combining a plurality of image recognition processes such as color recognition and shape recognition to perform person identification is conceivable. The problem remains that another person may be marked.

  In addition, with the recent adoption of IP for surveillance cameras, it has become a form where many camera images are concentrated and monitored on a single server, etc. via a network, and multiple recognitions such as face recognition and shape recognition There is a problem that the processing load is further increased by performing the processing.

  Therefore, the present invention provides a monitoring device, a monitoring method, and a monitoring method that can easily and reliably identify a marked person and display the mark without increasing the processing load of recognition processing even when there are a plurality of cameras. The purpose is to provide a program.

  In order to solve the conventional problem, the present invention performs marking on a monitoring target in a video imaged by a plurality of monitoring cameras based on an instruction from an operation unit such as a mouse, and the video from the monitoring camera. When a plurality of image recognition processes are performed to identify the marked monitoring target, the plurality of image recognition processes are limited according to the image recognition process result.

  According to the present invention, when a monitoring target is identified in a plurality of camera images, the plurality of image recognition processes are limited according to the result of the image recognition process. At this time, it is possible to control switching of the image recognition process, and even when there are a plurality of cameras, the marked person can be identified and displayed easily and reliably without increasing the processing load of the recognition process. In other words, for example, if there is a match with a monitoring target marked by a recognition process for a certain camera, only a more accurate recognition process will be performed for other camera images. However, if other recognition processing with low accuracy is not performed, there is an effect of preventing an unnecessary mark for specifying the monitoring target from being displayed. Also, for example, when image recognition processing is performed on a large number of camera images, when the target mark is displayed, if recognition processing for other camera images is not performed thereafter, image recognition is performed. There is an effect that the processing load can be suppressed.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  The embodiment of the present invention described below relates to a monitoring device that tracks and monitors a person who moves between a plurality of monitoring cameras as a monitoring target, and has found a suspicious person in a certain camera video. By marking in advance, the suspicious person can be prevented from losing sight by superimposing the mark display on the suspicious person on the monitor even after the suspicious person moves to another camera shooting range.

(Embodiment 1)
The person monitoring apparatus 100 in Embodiment 1 of this invention is demonstrated.

  FIG. 1 is an overall configuration diagram of a person monitoring apparatus 100 according to Embodiment 1 of the present invention.

  In FIG. 1, a person monitoring apparatus 100 according to Embodiment 1 is connected to a plurality of monitoring cameras (simply abbreviated as “cameras” in the drawing) 1 directly or via a network, An operation unit 3, a marking unit 4, a marking information storage unit 5, a person identification unit 6, and a recognition processing restriction unit 7 are included.

  The operation unit 3 receives an operation performed by a supervisor using a pointing device such as a mouse or a trackball.

  The marking unit 4 receives the operation of the monitor from the operation unit 3 and performs marking on a person such as a suspicious person designated by the monitor.

  The marking information storage unit 5 accumulates the feature amount of the person marked by the marking unit 4. The feature amount of the marked person may be extracted when the marking unit 4 or the like marks the suspicious person according to an instruction from the operation unit 3, or another new means is provided, Of course, the feature amount of the person marked by the marking unit 4 may be extracted.

  The person identification unit 6 determines whether or not they are the same person using a plurality of image recognition processes based on the video obtained from the monitoring camera 1 and the person feature amount obtained from the marking information storage unit 5. .

  The recognition process restriction unit 7 restricts the image recognition process used in the person identification unit 6 according to the image recognition process result of the person identification unit 6.

  The video processing unit 2 superimposes and displays a mark display on the person with the same feature amount on the video of the surveillance camera 1 in accordance with the image recognition processing result of the person identification unit 6.

  The display unit 8 receives a video being shot from the video processing unit 2 and displays the video on a display device such as a monitor or a display.

  FIG. 2 is a diagram showing a usage image of the person monitoring apparatus 100 according to the first embodiment shown in FIG.

  The monitor uses the pointer 31 to mark the suspicious person 22 reflected in the video display screen 21 of the camera A 161 of the display unit 8. At that time, the suspicious person 22 is not shown in the video display screen 21 of the camera B162. Thereafter, it is assumed that the suspicious person 22 goes out of the shooting range of the camera A161, moves along the path 160, and enters the shooting range of the camera C163 after 5 minutes. At this time, the monitoring device compares the person currently shown with the person marked by the monitor. If it is determined that the person is the same person, on the video display screen 25 of the camera C 163 of the display unit 8, the person 22 is marked with a confirmation mark 24, thereby indicating that the person is marked with respect to the supervisor. Is done.

  The operation of the person monitoring apparatus 100 configured as described above will be described below.

  Each of the plurality of surveillance cameras 1 is connected to the person monitoring apparatus 100 directly or indirectly via a network such as the Internet or a LAN, and continuously distributes images being captured.

  The video distributed by the monitoring camera 1 is passed to the video processing unit 2 and the person identification unit 6 of the person monitoring apparatus 100. Normally, the video processing unit 2 passes the received video as it is to the display unit 8, and the display unit 8 displays it on a display device such as a monitor in real time.

  The person identification unit 6 performs person identification processing based on the marking information by the marking unit 4 stored in the marking information storage unit 5.

  Here, first, the operation of the marking unit 4 will be described with reference to FIG.

  In FIG. 3A, the monitor operates the operation unit 3 for the persons 22 and 23 shown in the video display screen 21, and selects and selects the person to be tracked, for example, the person 22 by using the pointer 31. To do. Then, the marking unit 4 extracts a person feature amount from the marked video of the person 22 using an image recognition process, and accumulates it in the marking information storage unit 5.

  Note that the method of extracting the feature information for each person in the marking unit 4 is a known method, and is described in detail in, for example, “Image Processing and Recognition” (authored by Takeshi Aoi, Tomoharu Nagao, Shoshodo Publishing). Are listed. Therefore, a detailed description of the procedure for extracting the person feature amount is omitted.

  The marking information storage unit 5 is configured by a storage device or a database, and accumulates the person feature amount acquired by the marking unit 4. Examples of the person feature amount included in the marking information storage unit 5 include a face feature amount, a color feature amount, a shape feature amount, and the like. As feature information for specifying an object by shape, color, size, etc., for example, a face There is information for specifying the shape and position of the eyes, nose and mouth in the case of the feature amount.

  The person identification unit 6 keeps tracking the person during the period in which the person 22 is shown in the same video display screen 21. As a method of tracking a person, a known method as described below may be used.

  That is, first, an object is extracted from the video of the monitoring camera 1 using the background difference, and the correlation between the object positions is examined between the previous and subsequent frames. It is determined that objects with close object positions are the same object, and movement trajectory information of the object is generated to track the person. Then, the position of the person who should display the mark being tracked and the type of the mark are transmitted to the video processing unit 2.

  The video processing unit 2 receives the video being imaged from the monitoring camera 1 and receives the position of the person being tracked in the video and the type of mark when displaying the mark from the person identification unit 6. Then, as shown in FIG. 3B, a screen in which the confirmation mark 24 is superimposed on the person 22 on the video display screen 21 of the monitoring camera 1 is generated and transmitted to the display unit 8.

  When the person 22 moves from the camera A to another camera, the person identifying unit 6 performs person identification using image recognition processing based on the images acquired from the marking information storage unit 5 and the monitoring camera 1. The details of the person identification unit 6 will be described in detail later, and thus the description thereof is omitted here.

  In FIG. 3C, if the person 26 shown on the video display screen 25 of the camera different from the video display screen 21 is the same person as the marked person 22 in FIGS. 3A and 3B. Are displayed in a state specified by the confirmation mark 24.

  Further, when the determination result of the person identification unit 6 is low in accuracy and it is uncertain whether or not it is the same person as the marked person 22, an uncertain mark 27 as shown in FIG. Is displayed.

  Next, the detailed structure of the person identification part 6 is demonstrated using FIG.

  FIG. 4 shows a detailed configuration example of the person identification unit 6 of the person monitoring apparatus 100 according to the first embodiment.

  In FIG. 4, the person identifying unit 6 according to the first embodiment includes a detection mode-specific recognition switching processing unit 61, a face recognition processing unit 62, a color recognition processing unit 63, a shape recognition processing unit 64, and a person specifying process. Part 65.

  The detection mode-specific recognition switching processing unit 61 is based on the detection mode acquired from the recognition processing restriction unit 7 and performs a plurality of image recognition processes performed on the image received from the monitoring camera 1, that is, the face recognition processing unit 62, color recognition. The processing unit 63 and the shape recognition processing unit 64 are switched.

  The face recognition processing unit 62 extracts a face feature amount from the image received from the detection mode-specific recognition switching processing unit 61 and evaluates whether or not it matches the face feature amount in the marking information storage unit 5.

  The color recognition processing unit 63 extracts a color feature amount from the image received from the detection mode-specific recognition switching processing unit 61 and evaluates whether it matches the color feature amount in the marking information storage unit 5.

  The shape recognition processing unit 64 extracts a shape feature amount from the image received from the recognition mode-specific recognition switching processing unit 61 and evaluates whether or not it matches the shape feature amount in the marking information storage unit 5.

  The person identification processing unit 65 inputs a photographed video from the camera 1 and receives evaluation results from the face recognition processing unit 62, the color recognition processing unit 63, and the shape recognition processing unit 64. If the evaluation result is sufficiently high, the video processing unit 2, the corresponding person is displayed as a mark based on the marking information stored in the marking information storage unit 5, the recognition process evaluation unit 7 is fed back to the recognition process evaluation result, and the mark is displayed. The tracking is continued until the person and the person marked by the marking unit 4 move out of the camera photographing range.

  In the first embodiment, the face recognition processing unit 62 is capable of performing person identification determination with higher accuracy than other recognition processing, and the color recognition processing unit 63 and the shape recognition processing unit 64 are face recognition processing. Although it is inferior in accuracy to the unit 62, it is possible to make a person identification determination using an image in which no face is shown. In addition, the evaluation result of each recognition process is ranked according to the threshold value according to the threshold value, and is regarded as coincidence, semi-coincidence, or disagreement.

  FIG. 5 is a table showing an example of transition of the person detection mode in the recognition processing restriction unit 7 of the present embodiment.

  As shown in FIG. 5, the recognition processing restriction unit 7 has five types of determination outputs A to E as the recognition result determination processing, and prepares three types of detection modes and mark displays, respectively.

  That is, for example, when “A; face recognition match” is output as the recognition result determination process, the detection mode after transition is “non-detection mode”, and the mark display is “confirmation mark display”.

  Similarly, when “B; face recognition semi-match” is output as the recognition result determination process, the detection mode after the transition is “face detection mode”, the mark display is “indeterminate mark display”, and the recognition result determination process is performed. When “C; color or shape recognition coincidence” is output, the detection mode after the transition is “face detection mode”, and the mark display is “indeterminate mark display”.

  Similarly, when “D; face mismatch and color or shape recognition semi-match” is output as the recognition result determination process, the detection mode after transition is “all detection mode” and the mark display is “indeterminate mark display”. Thus, when “E; all mismatch” is output as the recognition result determination process, the detection mode after transition is “all detection mode”, and no mark is displayed.

  FIG. 6 is a diagram illustrating a transition of the person detection mode in the recognition processing restriction unit 7 of the present embodiment.

  In FIG. 6, the all detection mode 701 is an initial state and performs recognition processing for all faces, colors, and shapes. In the face detection mode 702, only the face recognition processing unit 62 is performed. Further, the non-detection mode 703 does not perform all recognition processing.

  The above three person detection modes exist individually for the person registered in the marking information storage unit 5, and different detection modes can be taken for each marked person.

  The person identification unit 6 restricts the recognition process executed when comparing the person shown in the images of the plurality of surveillance cameras 1 with the marked person according to the detection mode for each marked person.

  That is, as shown in FIG. 6, in the all detection mode 701, when the face recognition results are inconsistent and any of the recognition processing results other than the face is semi-matched, the route 710 is used. Return to all detection mode 701. At this time, the person identification processing unit 65 instructs the video processing unit 2 to display the uncertain mark 27 for the person as shown in FIG. Also, at this time, if it is determined that the person is the same person already marked in the other camera image and there is an uncertain mark 27 of the person being tracked, only the one with the higher evaluation value is displayed, An indeterminate mark is displayed for a plurality of persons (for example, in the case of the recognition result determination process in FIG. 5D).

  In the all detection mode 701, when the result of the face recognition process is a semi-match, or when the result of the face recognition is a mismatch and any of the results other than the face is a match, as shown in FIG. The face transitions to the face detection mode via the path 711. This is because face recognition is more accurate than other recognition processes, so if a half-match occurs due to face recognition, even if a match occurs due to other recognition processes that are less accurate than face recognition, This is because there is no need to continuously recognize other than the face. Similarly, when a match occurs due to recognition other than the face, it is only necessary to continue the determination by the face recognition process, which is a process with higher accuracy than the color recognition and shape recognition. At this time, the person identification processing unit 65 instructs the video processing unit 2 to display the uncertain mark 27 for the person as shown in FIG. 7B (for example, FIG. 5B or FIG. 5C). For recognition result judgment processing).

  In the all detection mode 701, when the result of the face recognition process is coincident, the mode transits to the non-detection mode 703 via a path 714 as shown in FIG. At this time, the person identification processing unit 65 instructs the video processing unit 2 to display the confirmation mark 24 for the person as shown in FIG. Further, it is determined that the person is the same person as the person marked in the other camera image, and if there is an indefinite mark of the person being traced and displayed, it is cleared and the tracking display is stopped (for example, the recognition result of FIG. 5A). For judgment processing).

  In the face detection mode 702, when the result of the face recognition process is a half match, the process returns to the face detection mode 702 via a path 712 as shown in FIG. At this time, the person identification processing unit 65 instructs the video processing unit 2 to display the indeterminate mark 27 for the person as shown in FIG. Also, at this time, if it is determined that the person is the same person already marked in the other camera image and there is an uncertain mark 27 of the person being tracked, only the one with the higher evaluation value is displayed, A plurality of uncertain marks are displayed (for example, in the case of the recognition result determination process in FIG. 5B).

  In the face detection mode 702, when the result of the face recognition process is coincident, the mode transits to the non-detection mode 703 via a path 713 as shown in FIG. At this time, the person identification processing unit 65 instructs the video processing unit 2 to display the confirmation mark 24 for the person as shown in FIG. Also, at this time, it is determined that the person is the same person already marked in the other camera image, and if there is an indeterminate mark of the person being tracked, it is cleared and the tracking display is stopped (for example, (In the case of recognition result determination processing in FIG. 5A).

  Note that in the face detection mode 702 and the non-detection mode 703, when all of the persons displayed with the confirmation mark and the unconfirmation mark tracked by the person identification processing unit 65 have moved out of the camera, FIG. As shown in FIG. 5, a transition is made to the all detection mode 701 via the path 716 or the path 715, respectively (for example, in the case of the recognition result determination process in FIG. 5E). That is, it returns to the initial state.

  Next, the operation of the person identification unit 6 will be described using the processing flow of the person identification unit 6 shown in FIG.

  FIG. 9 is a flowchart showing a processing procedure of the person identification unit 6.

  In the person identification unit 6, as shown in FIG. 4, first, the detection mode-specific recognition switching processing unit 61 receives detection modes for all marked persons from the recognition processing restriction unit 7 and determines whether or not to perform person detection. (S101). Here, when the detection mode of all the marked persons is the non-detection mode 703, the recognition mode-specific recognition switching processing unit 61 determines that no person is detected (S101 “NO”), and the subsequent steps Recognition processing is not performed.

  On the other hand, when the detection mode includes a mode other than the non-detection mode 703 (S101 “YES”), the recognition mode-specific recognition switching processing unit 61 acquires a camera video from the monitoring camera 1 (S102).

  When a person is not included in the video acquired in S102 (S103 “no person”), the detection mode-specific recognition switching processing unit 61 returns to the top of the flow (S103).

  When a person is included in the video acquired in S102 (S103 “has a person”) and the detection mode includes the all detection mode 701 (S104 “all detection”), the recognition mode processing switching unit 61 for each detection mode. Acquires from the marking information storage unit 5 a feature quantity such as the color and shape of a marked person whose detection mode is the all detection mode 701 (S105), and the color recognition processing unit 63 and the shape recognition processing unit 64 The color recognition process or the shape recognition process is executed on the person in the video acquired in S102 based on the acquired feature amount such as the color or shape of the person, and the person identification determination process based on the shape or the color is performed. (S106), the process proceeds to S107.

  If it is determined in S104 that the detection mode is all face detection mode 702 (S104 “face detection”), the process proceeds to S107 without performing S105 and S106.

  In S107, the facial feature amount of the marked person whose detection mode is other than the non-detection mode 703 is acquired from the marking information storage unit 5, and the face recognition processing unit 62 is applied to the person in the video acquired in S102. Thus, the face recognition process is executed, and the person identification determination process based on the face is performed (S108).

  The person identification processing unit 65 receives the recognition process result of the person identification determination process of S106 and S108, and determines whether the result is a match, a semi-match, or a mismatch (S109).

  When there is a match and a semi-match (S109 “YES”), the person identification processing unit 65 determines which match is made in which recognition process between the marked person and the person in the video with respect to the recognition process restriction unit 7. The recognition processing restriction unit 7 switches the detection mode of the marked person according to the state transition shown in FIG. 6 based on the current detection mode and the information received from the person identification processing unit 65 ( S110).

  The recognition processing restriction unit 7 then displays the position of the person on the screen and the type of the mark so that the video processing unit 2 performs marking display on the person in the video that matches or semi-matches. (Confirmation mark, unconfirmation mark) is notified and a mark is displayed to the person (S111). Indeterminate marks may be displayed on a plurality of persons at the same time.

  If there is a mismatch in the determination in S109 (S109 “NO”) and after the above process is completed, the process returns to S101 again, and the recognition process for the camera video is repeated.

  Next, a specific example of mark display on a person will be described.

  FIG. 10 is a diagram illustrating a specific example of mark display on a person.

  For example, it is assumed that the person 22 shown in FIG. 10A is marked and registered in the marking information storage unit 5. In the camera A of FIG. 10B and the camera B of FIG. 10C, when there is a match due to shape recognition for the person 22 and the person 23, the indeterminate mark 27 is displayed simultaneously for both persons.

  At this time, although the person 22 and the person 23 are both highly likely to be registered in the marking information storage unit 5, they indicate that they are not confirmed.

  Then, when the coincidence with the person marked by the person identification determination process thereafter is confirmed, the other uncertain marks 27 are deleted.

  FIG. 11 is a diagram illustrating another specific example of displaying a mark on a person.

  For example, in FIGS. 11A and 11C, the person 22 and the person 23 are matched by shape recognition, and an indeterminate mark 27 is displayed. At this time, the detection mode transitions to the face detection mode. Thereafter, when the person 22 moves and a face image can be acquired and a match by face recognition occurs, the indeterminate mark 27 changes to a confirmed mark 24 as shown in FIG. Further, since the person 23 is not a marked person, the indeterminate mark 27 attached to the person 23 is deleted as shown in FIG. At this time, the detection mode transitions to the non-detection mode. Thereafter, while the person 22 exists in the camera, the person identification determination process for the person 22 is performed in the person identification unit 6 of the camera A and the camera B. Absent.

  Note that the indeterminate mark 27 may change the display method of its shape, color, etc. according to the evaluation value of the image recognition result.

  FIG. 12 is a diagram showing another specific example of displaying a mark on a person.

  For example, in FIG. 12A, the color density of the indeterminate mark 210 is changed according to the height of the evaluation value. That is, the lighter the color, the lower the evaluation value, and the darker the color, the higher the evaluation value.

  Further, for example, in FIG. 12B, the size of the indeterminate mark 211 is changed according to the height of the evaluation value. That is, the smaller the size, the lower the evaluation value, and the larger the size, the higher the evaluation value.

  FIG. 13 is a diagram showing another specific example of displaying a mark on a person.

  Further, in FIG. 13, the shape of the indeterminate mark 212 is changed according to the type of recognition processing that has been matched.

  For example, the inverted triangle indeterminate mark 212 in FIG. 13A indicates a semi-coincidence of face recognition, and the diamond indeterminate mark 213 in FIG. 13B indicates a match by color recognition. The arrow-shaped uncertain mark 214 in FIG. 13C indicates a match by shape recognition.

  By using the above display method, the supervisor can visually know the evaluation value of the person identification determination and the type of recognition that coincides.

  As shown in FIG. 13D, numerals may be displayed on the confirmation mark 24 and the uncertain mark 27 as shown in FIG. The meaning of the numbers may be, for example, the total number of marks when a plurality of uncertain marks 27 are displayed, or the order in which marks are added. In addition, the ID number assigned to the marked person, the ID number assigned to the camera in which the person was photographed at the time of marking, or the case where a plurality of supervisors can mark, It may be an assigned ID number.

  By using the above display method, when a mark is displayed on a person, the work for checking the information of the person can be performed efficiently by referring to the numbers.

  Instead of the camera 1 having the configuration shown in FIG. 1, the video recorded on a storage device such as an HDD is read out and transferred to the person identification unit 6 and the video processing unit 2, and the person is marked on the video. The tracking display may be performed. In this case, when playing back images from a plurality of cameras from the storage device, it is desirable to play back the images synchronized with each other.

  In addition, image recognition processing is performed on the stored video in advance, and the face feature value, color feature value, and shape feature value of each person are recorded together in the storage device, so that the person identification unit 6 can display an image during playback. It is not necessary to perform recognition processing, and person identification can be performed only by comparison with the feature amount in the marking information storage unit 5. Therefore, person tracking can be performed with a reduced processing load.

  Also, if the movement trajectory information of a person is accumulated in advance, for example, when a person enters the shooting range of the camera, it is initially facing backwards, and a face appears after 10 seconds and a facial feature amount is obtained. By referring to the movement trajectory information accumulated at this time, it is possible to perform person identification processing by comparing facial feature amounts immediately after the person enters the camera shooting range. can do.

  As described above, according to the person monitoring apparatus 100 of Embodiment 1 of the present invention, by providing the recognition processing restriction unit 7, it is possible to perform switching control of image recognition processing according to the detection mode for each camera image.

  For this reason, when there is a match between people in a certain camera recognition process, only a more accurate recognition process is performed on other camera images, and other low-accuracy recognition processes are not performed. This has the effect of preventing an unnecessary mark for identifying a person from being displayed.

  Also, when image recognition processing is performed on a large number of camera images, when the marked person is displayed, the recognition processing for other camera images is not performed, so that the images of a plurality of cameras can be integrated. In the configuration in which the image recognition processing is performed by collecting the data in one server, an effect of suppressing the load of the image recognition processing can be obtained.

(Embodiment 2)
Next, the person monitoring apparatus 200 in Embodiment 2 of this invention is demonstrated.

  FIG. 14 is a diagram illustrating a configuration of the person monitoring apparatus 200 according to the second embodiment.

  In the person monitoring apparatus 100 according to the first embodiment described above, as shown in FIG. 1, the video processing unit 2, the marking information storage unit 5, the person identification unit 6, and the recognition processing restriction unit 7 are all configured on the same server. However, in the person monitoring apparatus 200 according to the second embodiment, the video processing unit 2 and the person identification unit 6 are different from the server 93 having the recognition processing restriction unit 7 and the marking information storage unit 5. 91 and 92 are connected by a network such as a LAN or the Internet.

  In addition, each person identification unit 6 performs person identification on the video from the monitoring cameras 11 and 12 connected to the servers 91 and 92 including the person identification units 6.

  The operation of the person monitoring apparatus 200 according to the second embodiment configured as described above will be described below with an example.

  FIG. 15 is a diagram showing a specific example of mark display on a person in the second embodiment.

  When marking is performed on the person 22 in the video display screen 28 within the shooting range of the monitoring camera 11 shown in FIG. 15A by operating the operation unit 3, marking information is stored in the marking information storage unit 5 of the server 93. Is accumulated.

  Thereafter, based on the marking information stored in the marking information storage unit 5 of the server 93, the person identification unit 6 of the server 91 compares the person in the video of the monitoring camera 11 and at the same time identifies the person of the server 92. The unit 6 also compares with the person in the video of the surveillance camera 12. At this time, both the person identification units 6 of the servers 91 and 92 control the recognition process used for person identification according to the detection mode of the recognition process restriction unit 7 of the server 93.

  Thereafter, it is assumed that the person 22 has moved into the video display screen 29 of the shooting range of the monitoring camera 12. When the face image of the person 22 is taken by the monitoring camera 12, the person identifying unit 6 of the server 92 matches the face feature amount of the person 22 with the face feature amount of the marking information storage unit 5, and is the same person. It is determined. Next, in the video processing unit 2 of the server 92, a confirmation mark 24 as shown in FIG. 15B is displayed on the person 22 in the video of the monitoring camera 12, and the marked person is displayed as a mark. Further, the detection mode of the recognition processing restriction unit 7 of the server 93 transitions to the non-detection mode 703.

  Therefore, the person identification unit 6 of the server 91 stops the comparison process between the person 22 and the person in the monitoring camera 11 according to the detection mode changed by the person identification unit 6 of the server 92. This is because it is determined that the marked person 22 is within the shooting range of the monitoring camera 12 and at the same time cannot be within the shooting range of the monitoring camera 11. Therefore, the server 91 can prevent unnecessary processing of the person identification unit 6 from being executed.

  Thereafter, when the person 22 moves out of the shooting range of the monitoring camera 12, the person identification unit 6 of the server 92 changes the detection mode inside the recognition processing restriction unit 7 of the server 93 to the all detection mode 701. Therefore, the person identifying unit 6 of the server 91 resumes the comparison process with the person in the monitoring camera 11 again.

  Since other processes are the same as those in the first embodiment, detailed description thereof is omitted.

  Note that the video processing unit 2 and the person identification unit 6 may be configured inside the monitoring camera 11 and the monitoring camera 12 as in the person monitoring device 300 shown in FIG. In FIG. 16, each person identification unit 6 performs person identification on the video from the imaging means 13 of the monitoring cameras 11 and 12 including the person identification unit 6. The operation at this time is the same as the operation in the configuration shown in FIG.

  As described above, according to the person monitoring apparatuses 200 and 300 according to the second embodiment of the present invention, even when the monitoring system is increased in scale and the number of cameras is increased, the person identification unit 6 is installed in a plurality of servers. It is possible to construct a system that performs person tracking while cooperating with. Distributing the load of image recognition eliminates the need for a high-performance server and has the effect of reducing the construction cost of the person tracking system at a low price.

  Further, according to the configuration shown in FIG. 16, by installing the person identification unit 6 on each of the monitoring cameras 11 and 12, there is an effect of distributing the processing load on the server side to the monitoring cameras 11 and 12 side. Further, since each person identifying unit 6 only needs to perform image recognition processing on the video of each of the registered surveillance cameras 11 and 12, a high-speed CPU is not required, and costs can be reduced.

  In any of the above configurations, the recognition processing restriction unit 7 performs recognition processing control according to the determination result of the person identification unit 6 mounted on each server 91, 92, 93 or each monitoring camera 11, 12. Thus, there is an effect of suppressing the processing load of the entire system by canceling unnecessary recognition processing.

(Embodiment 3)
Next, the operation of the operation unit 3 with respect to the display unit 8 in the third embodiment of the present invention will be described.

  FIG. 17 is a diagram illustrating an operation screen when the indeterminate mark 27 is selected by the supervisor.

  The monitor can operate the pointer 31 to select the indeterminate mark 27 and use the menu 32 to change or refer to the marking information storage unit 5.

  When the mark release 321 is selected from the menu 32, the indeterminate mark 27 is deleted. Then, the person 22 is registered in the marking information storage unit 5 as a person different from the person determined to match. Hereinafter, this information is referred to as mismatch information. The person identification processing unit 65 of the person identification unit 6 does not determine as a marked person when the feature amount matches or is sufficiently close to the person registered as the mismatch information. Therefore, the person 22 is not displayed as a mark thereafter. Further, the detection mode of the recognition processing restriction unit 7 transitions to the all detection mode 701.

  When the confirmation mark 322 is selected, the uncertain mark 27 is changed to the confirmation mark 24. At this time, if there is another person on which the indefinite mark 27 is displayed, it is deleted. Further, the detection mode of the recognition processing restriction unit 7 transitions to the non-detection mode 703.

  When the person search 323 is selected, a picture of the person is searched from the recorded pictures of the person and displayed.

  When the marking information display 324 is selected, the person information stored in the marking information storage unit 5 can be referred to. For example, the face image of the person is displayed as, for example, the face image display 325, while information such as the time and place marked by the supervisor is displayed as the marking location 326.

  Further, when the priority recognition change 327 is selected, it is possible to change the order of recognition processing to be emphasized at the time of person identification. For example, if the color of the clothes worn by the marked person is very characteristic, the accuracy of tracking and displaying the person is improved by setting the color recognition result more important than the face recognition. .

  FIG. 18 is a diagram showing an operation screen when the confirmation mark 24 is selected by the supervisor.

  When the mark release 331 is selected from the menu 33, the confirmation mark 24 is deleted. Then, the person 22 is registered in the marking information storage unit 5 as a person different from the person determined to match. Hereinafter, this information is referred to as mismatch information. The person identification processing unit 65 of the person identification unit 6 does not determine as a marked person when the feature amount matches or is sufficiently close to the person registered as the mismatch information. Therefore, the person 22 is not displayed as a mark thereafter. In addition, the detection mode of the recognition processing restriction unit 7 transitions to the all detection mode 701.

  When the uncertain mark 332 is selected, the confirm mark 24 is changed to the uncertain mark 27. At this time, the detection mode of the recognition processing restriction unit 7 transitions to the face detection mode 702.

  The person search 333, marking information display 334, face image display 335, marking place 336, and priority recognition change 337 are the person search 323, marking information display 324, face image display 325, marking place 326 described in FIG. It is equal to the priority recognition change 327.

  FIG. 19 is a diagram showing an operation screen when a person who is not marked is selected.

  The monitor can operate the pointer 31 to select the person 22 and use the menu 34 to perform new marking or person search.

  When a new marking 341 is selected from the menu 34, the person obtains a feature amount in the marking unit 4 and is registered in the marking information storage unit 5.

  When the marking change 343 is selected, the marking information 344 of the person registered in the marking information storage unit 5, for example, “marking information 1”, “marking information 2”, “marking information 3”,... Is displayed. The supervisor can manually select the marking information 344 of the person who seems to match the person 22 from among them, and display the mark on the person 22. At this time, the confirmation mark 24 is displayed on the person 22, and the detection mode of the recognition processing restriction unit 7 transitions to the non-detection mode 703. At this time, if the confirmation mark 24 has already been displayed by the person identification unit 6 for another person, the manual operation of the supervisor is given priority. For this reason, the confirmation mark 24 automatically attached disappears.

  The person search 342 is the same as that described above.

  As described above, according to the third embodiment, the monitor can change the mark attached to the monitoring device and change the detection mode of the recognition processing restriction unit 7 by his / her own operation. And by acquiring the operation result, there exists an effect which the precision of the person identification part 6 improves. For example, for a person who is registered in the discrepancy information by canceling the mark, an incorrect mark is not displayed thereafter. In addition, by performing priority recognition change, it is possible to identify a person with high accuracy for a person having a characteristic in color or shape.

  In the first to third embodiments, the person monitoring devices 100, 200, and 300 are described as hardware (circuit) as shown in FIG. 1 and the like. However, the present invention is not limited to this. For example, a software program, that is, a monitoring program is stored in a storage unit such as a memory or a hard disk in the person monitoring device 100, 200, 300, and the CPU or the like reads and executes the program. Of course.

  In the first to third embodiments described above, a person is a monitoring target and a mark is displayed on the person. However, the present invention is not limited to this. Of course, a moving machine such as a robot may be included.

  According to the monitoring device, the monitoring method, and the monitoring program according to the present invention, when a monitoring target is identified in a plurality of camera images, the plurality of image recognition processes are limited according to the result of the image recognition process. When identifying the monitoring target for multiple camera images, it is possible to control the switching of the image recognition process, and even if there are multiple cameras, it is easy to identify a marked person without increasing the processing load of the recognition process It has the advantageous effect of being able to identify and display the mark reliably, for example, by installing multiple surveillance cameras in a wide range of department stores, shopping malls, amusement parks, etc. It is effective for a monitoring device that tracks the behavior of a specific person.

The figure which shows the structure of the person monitoring apparatus in Embodiment 1 of this invention. The figure which shows the usage image of the person monitoring apparatus in Embodiment 1 of this invention The figure which shows the example of the person marking method and video display screen in Embodiment 1 of this invention The figure which shows the structure of the person identification part in Embodiment 1 of this invention. The figure which showed the state transition of the detection mode of the recognition process restriction | limiting part in Embodiment 1 of this invention with the table | surface The figure which shows the state transition of the detection mode of the recognition process restriction | limiting part in Embodiment 1 of this invention. The figure which shows operation | movement when the indefinite mark is displayed in Embodiment 1 of this invention. The figure which shows operation | movement when the confirmation mark in Embodiment 1 of this invention is displayed. The flowchart which shows operation | movement of the person identification part in Embodiment 1 of this invention. The figure which shows the example in which multiple indefinite marks are displayed on the video display screen of the display part in Embodiment 1 of this invention. The figure which shows the example in which an indefinite mark changes to another mark in the video display screen of the display part in Embodiment 1 of this invention. The figure which shows the example which added the change to the color and magnitude | size of an indeterminate mark in Embodiment 1 of this invention The figure which shows the example which added the change to the shape of the indeterminate mark in Embodiment 1 of this invention The figure which shows the structure of the person monitoring apparatus in Embodiment 2 of this invention. The figure which shows the example of the video display screen of the display part in Embodiment 2 of this invention. The figure which shows the structure of the person monitoring apparatus which has arrange | positioned the person identification part in the camera side in Embodiment 2 of this invention The figure which shows the operation screen at the time of the indeterminate mark selection in Embodiment 3 of this invention. The figure which shows the operation screen at the time of the confirmation mark selection in Embodiment 3 of this invention. The figure which shows the operation screen at the time of selection of a person without a mark in Embodiment 3 of this invention

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,11,12 Surveillance camera 2 Image | video processing part 3 Operation part 4 Marking part 5 Marking information storage part 6 Person identification part 7 Recognition process restriction | limiting part 8 Display part 13 Imaging means 21, 25, 28, 29 Image | video display screen 22,23 , 26 Person 24 Confirmation mark 27 Uncertain mark 100, 200, 300 Person monitoring device 210, 211, 212, 213, 214 Uncertain mark 31 Pointer 32 Menu when uncertain mark is selected 321, 331 Mark release button 322 Confirm mark Button 323, 333, 342 Person search button 324, 334 Marking information display button 325, 335 Face image display button 326, 336 Marking location 327, 337 Priority recognition change 33 Menu at confirmation mark selection 332 Uncertain mark button 34 Person selection Menu of time 341 New marking button 343 Marking change button 344 Marking information selection button 61 Recognition switching processing unit for each detection mode 62 Face recognition processing unit 63 Color recognition processing unit 64 Shape recognition processing unit 65 Human identification processing unit 701 All detection mode 702 Face detection mode 703 Non-detection mode 710, 711, 712, 713, 714, 715, 716 State transition path 91, 92, 93 Server

Claims (10)

A marking unit that performs marking on a monitoring target in a video from a plurality of monitoring cameras based on an instruction from an operation unit such as a mouse;
An identification unit for identifying the marked monitoring target by performing a plurality of image recognition processes on the video from the monitoring camera;
A recognition processing restriction unit for restricting the plurality of image recognition processes used in the identification unit according to an image recognition processing result by the identification unit;
A video processing unit that superimposes a mark display that identifies the monitoring target identified by the identification unit on the video from the monitoring camera;
Monitoring device. Furthermore, a marking information storage unit that stores the feature quantity of the monitoring target marked by the marking unit,
The identification unit identifies the monitoring target by performing a plurality of image recognition processes on the video from the monitoring camera based on the feature amount of the monitoring target.
The monitoring device according to claim 1. The identification unit and the video processing unit are provided in a device different from the recognition processing restriction unit, and these devices are connected by a network.
The monitoring apparatus according to claim 1. The recognition processing restriction unit
When restricting the plurality of image recognition processes used in the identification unit executed for each video from a plurality of monitoring cameras, the accuracy of the image recognition processing matched when the monitoring target is identified in a certain monitoring camera video is defined as a standard. As described above, the image recognition processing with higher accuracy is continuously executed for the other surveillance camera images, and the image recognition processing with lower accuracy for the other surveillance camera images is stopped to be stopped, and the mark is already displayed. If it is done, cancel the mark display,
The monitoring device according to any one of claims 1 to 3. The recognition processing restriction unit
When the monitor receives an operation for releasing the mark display for the monitoring target displayed on the monitor of the display unit from the operation unit, the restriction on the image recognition process used in the identification unit is canceled.
The monitoring device according to any one of claims 1 to 3. The recognition processing restriction unit
When the monitor receives an operation for canceling the mark display for the monitoring target displayed on the monitor of the display unit from the operation unit, the marking information storage unit uses the information of the monitoring target displayed as a mark as mismatch information. And controlling the identification unit to use the mismatch information when identifying the monitoring target.
The monitoring device according to any one of claims 1 to 3. The recognition processing restriction unit
The image recognition process used in the identification unit when the monitor receives an operation for selecting a priority image recognition process by specifying a mark display for the monitoring target displayed on the monitor of the display unit from the operation unit Change the priority of
The monitoring device according to any one of claims 1 to 3. The recognition processing restriction unit
From the operation unit, the monitor designated the monitoring target displayed on the monitor of the display unit, and received an operation of manually associating the monitoring target information registered in the marking information storage unit with the monitoring target In this case, the image recognition process used in the identification unit is limited to be stopped.
The monitoring device according to any one of claims 1 to 3.   Marking is performed on a monitoring target in a video imaged by a plurality of monitoring cameras based on an instruction from an operation unit such as a mouse, and the marking is performed by performing a plurality of image recognition processes on the video from the monitoring camera. A monitoring method for restricting the plurality of image recognition processes according to a result of the image recognition process when identifying a monitoring target. On the computer,
Marking is performed on a monitoring target in a video imaged by a plurality of monitoring cameras based on an instruction from an operation unit such as a mouse, and the marking is performed by performing a plurality of image recognition processes on the video from the monitoring camera. A monitoring program for restricting the plurality of image recognition processes in accordance with the image recognition process result when identifying a monitoring target.

Images