JP2016021716A - Tracking device and control method of the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique for efficiently tracking a monitoring object even if the number of monitoring objects increases.SOLUTION: When the number of subjects recognized from a captured image is equal to or larger than a threshold, a subject group belonging to one group in subject groups recognized from the captured image is set to be a tracking object. When the number is less than the threshold, the respective subjects recognized from the captured image are set to be the tracking objects.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing technique, and more particularly to a technique for tracking a moving body from images taken by a plurality of surveillance cameras.

  In recent years, as a monitoring and monitoring camera system, a system for tracking a monitoring target over a wide area has been considered with an increase in the number of installed monitoring cameras. For example, Patent Document 1 proposes a system that automatically recognizes and tracks a moving body that moves over a wide range across a large number of cameras. In addition, when monitoring is performed over a wide area, the monitoring target may act as a group. However, in the technique disclosed in Patent Document 2, a person acts as a single person by grouping persons based on information on the behavior of the person. Detecting whether you are doing group action.

JP 2007-142527 JP 2006-92396 JP

  However, in the above-described system, in the environment where a plurality of cameras are installed in the wide-area monitoring range, when a person to be tracked as a monitoring target increases, tracking is performed for a specific monitoring target moving in the area. This increases the load of processing for searching and collating. In addition, when the monitoring target performs group action, the monitoring target person is likely to appear or hide, and it may be difficult to keep track of the monitoring target person.

  The present invention has been made in view of such a problem, and provides a technique for efficiently tracking a monitoring target even when the number of monitoring targets increases.

  One aspect of the present invention is a tracking device, and if the number of subjects recognized from a captured image is equal to or greater than a threshold, a subject group belonging to one group among the subject groups recognized from the captured image is set as a tracking target. If the number is less than a threshold value, a means for tracking each subject recognized from the captured image is provided.

  According to the configuration of the present invention, even if the number of monitoring targets increases, the monitoring targets can be tracked efficiently.

FIG. 2 is a block diagram illustrating an example of a functional configuration of the network camera 100. The figure which shows the structural example of a monitoring system. The flowchart of the process which the network cameras 100 and 200 perform. The figure which shows the example of arrangement | positioning of the network cameras 100 and 200 in real space. The figure explaining the process in step S6. The figure explaining the process in step S9.

  Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. The embodiment described below shows an example when the present invention is specifically implemented, and is one of the specific examples of the configurations described in the claims.

[First Embodiment]
First, a configuration example of the monitoring system according to the present embodiment will be described with reference to FIG.

  Each of the network cameras 100 and 200 is a monitoring camera arranged at a predetermined position in the real space, and is a tracking device that tracks a moving object or a human body, and picks up an image (moving image) of the real space within the shooting range. To do. Each of the network cameras 100 and 200 performs video analysis processing such as moving object detection, moving object tracking, human body detection, face recognition, and object detection on the video (captured image of each frame). The video analysis result is transmitted to the server apparatus 400 via the network 600.

  The server device 400 instructs each of the network cameras 100 and 200 to recognize / track, and stores the captured image and the video analysis result transmitted from the network camera 100 and 200 in the storage device 300.

  The display device 500 is a device that performs various displays based on captured images and video analysis results stored in the storage device 300. For example, a captured image of each frame by the network camera 100 and a captured image of each frame by the network camera 200 may be displayed on the display screen of the display device 500. Further, the video analysis result for the captured image may be superimposed and displayed on each captured image. The display device 500 may include an operation unit for searching for a captured image in which an event scene is captured from each captured image. The display device 500 is, for example, a device such as a general PC or a mobile terminal device. The display device 500 performs display as described above on the display screen of the device, and uses a touch panel screen, a keyboard, a mouse, or the like as the operation unit. To perform operation input.

  The network 600 is configured by a LAN, the Internet, or the like, and may be wireless or wired. The network 600 may be composed of two or more sub-networks. In this case, each sub-network is also composed of a LAN or the Internet, and may be wireless or wired. good.

  The configuration shown in FIG. 2 is merely an example of a configuration that can realize each process described below, and any configuration may be adopted as long as each process described below can be realized. Absent. For example, although the system shown in FIG. 2 has two network cameras as surveillance cameras, the number may be three or more, and in any case, each network camera operates in the same manner. become.

  Next, a functional configuration example of the network camera 100 that is a tracking device that tracks a person or a group will be described with reference to the block diagram of FIG.

  The control unit 199 (computer) executes processing using a computer program and data stored in a memory (not shown). Thereby, the control unit 199 performs operation control of each functional unit constituting the network camera 100, and executes or controls each process described later as what the network camera 100 performs.

  The imaging sensor unit 101 is an imaging element such as a CMOS that converts an optical image formed on the imaging surface into a digital electric signal by photoelectric conversion. The imaging sensor unit 101 converts the digital electric signal into a subsequent development processing unit 102. Is sent to.

  The development processing unit 102 generates a captured image such as an RGB image or a YUV image by performing pixel interpolation processing and color conversion processing using the digital electrical signal output from the imaging sensor unit 101. Further, the development processing unit 102 performs a predetermined calculation process using the captured image, and performs image processes such as white balance, sharpness, contrast, and color conversion based on the calculation result.

  That is, the imaging sensor unit 101 and the development processing unit 102 can capture a real-space video, and the development processing unit 102 outputs the captured video in units of frames.

  The person / group collation data storage unit 104 stores subject information and group information including subject groups. In this embodiment, a person is used as a subject, but other types of subjects may be used.

  The object information, that is, the person information (person information) includes, for example, information for recognizing the person (features of the person such as the face and clothes of the person) and identification information for the person ( For example, ID) is included.

  Group information, that is, group information composed of a plurality of persons (group information) includes, for example, identification information (for example, ID) of the group information, person information of each person constituting the group, The reference distance between each person constituting the group and the number of persons constituting the group (total number, sex ratio, etc.) are included.

  The server apparatus 400 performs new registration of person information and group information in the person / group collation data storage unit 104 and update of person information and group information already stored in the person / group collation data storage unit 104.

  When the person / group collation processing unit 103 receives the captured image from the development processing unit 102, first, the person / group collation processing unit 103 uses the person information stored in the person / group collation data storage unit 104 for each person in the captured image. Perform recognition processing. Then, the person / group matching processing unit 103 counts the number of recognized persons and determines whether or not the counted number is equal to or greater than a threshold value. This threshold is three people, for example. If the person / group matching processing unit 103 determines that the person is less than the threshold value, the person / group matching processing unit 103 sets the recognized person as a tracking target.

  On the other hand, if it is determined that the person / group matching processing unit 103 is equal to or greater than the threshold value, the person / group matching processing unit 103 uses one of the recognized persons using the group information stored in the person / group matching data storage unit 104. A group of persons belonging to the group is specified, and the specified group of persons (group) is set as a tracking target.

  The person / group verification processing unit 103 then encodes the captured image received from the development processing unit 102 and information (analysis information) indicating the tracking target specified by the above analysis result with respect to the captured image. Send to the unit 105.

  The video encoding unit 105 compresses and encodes the captured image received from the person / group matching processing unit 103 to generate a compressed encoded image. The compression encoding format is not limited to a specific format, and formats such as MPEG4, H.264, MJPEG, and JPEG are used. The video encoding unit 105 transmits the compression encoded image to the server device 400 via the LAN I / F unit 106 together with the analysis information received from the person / group matching processing unit 103.

  The server device 400 stores the compression-coded image and analysis information transmitted from each network camera including the network camera 100 in the storage device 300.

  The LAN I / F unit 106 functions as an interface for connecting the network camera 100 to the network 600, and the network camera 100 exchanges data with devices on the network 600 via the LAN I / F unit 106. Communication will be performed. Note that the LAN I / F unit 106 cooperates with the LAN I / F unit of the storage device 300 to construct a network file system such as NFS or CIFS, and stores the compressed encoded image and analysis information in the storage device. You may make it store in 300. FIG.

  Here, FIG. 4 shows an arrangement example of the network cameras 100 and 200 in the real space. FIG. 4 shows a floor showing the position of the network camera installed in the monitoring range and the flow of people. FIG. 4 shows a direction in which the network camera 100 as the first monitoring camera and the network camera 200 as the second monitoring camera are shooting with respect to the flow of people.

  Next, processing performed by each of the network cameras 100 and 200 that are tracking devices that track a person or a group will be described with reference to the flowchart of FIG. Hereinafter, the operation of the network camera 100 will be described as an example, but the network camera 200 operates in the same manner.

<Step S1>
First, the server apparatus 400 causes each network camera on the network 600 (in the present embodiment, the network cameras 100 and 200) to track the person information of the person tracked by the network camera and the network camera. Send group information for the group. When the control unit 199 of the network camera 100 receives the person information and group information transmitted from the server device 400 via the LAN I / F unit 106, the control unit 199 receives the received person information and group information from the person / group verification data storage unit. It stores in 104.

<Step S2>
When the person / group collation processing unit 103 receives the captured image from the development processing unit 102, the person / group collation processing unit 103 uses the person information stored in the person / group collation data storage unit 104 to recognize each person in the captured image. Execute.

<Step S3>
The person / group matching processing unit 103 counts the number of persons recognized from the captured image in step S2, and determines whether the counted number is equal to or greater than a threshold value. As a result of this determination, if the counted number is equal to or greater than the threshold, the process proceeds to step S7, and if the counted number is less than the threshold, the process proceeds to step S4. This threshold is three people, for example.

<Step S4>
If no one can be recognized in step S2, that is, if no person matching the person information is captured in the captured image, the process returns to step S2. On the other hand, when one or more persons are recognized from the captured image in step S2, that is, when one or more persons matching the person information are captured in the captured image, the person recognized from the captured image is set as a tracking target. The process proceeds to step S5.

<Step S5>
The video encoding unit 105 generates a compression-encoded image by performing compression encoding on the captured image that is the processing target in step S2. The video encoding unit 105 transmits the generated compressed code image and information (analysis information) related to the tracking target to the server apparatus 400 via the LAN I / F unit 106.

  This analysis information includes, for example, identification information in the person information used for recognizing the tracking target (that is, identification information of the person as the tracking target), the position of the tracking target in the captured image (the person's recognition from the captured image). Area location). However, any information may be adopted as analysis information as long as it is information for specifying a person who has been successfully recognized in the captured image.

  Each time the server apparatus 400 receives a compressed code image and analysis information from the network camera 100, the server apparatus 400 stores the set of the compressed code image and analysis information in the storage device 300. As a result, for each network camera, a set group of compressed code images and analysis information transmitted from the network camera is registered in the storage device 300.

  Further, the network camera 100 controls the angle of view and orientation of the network camera 100 so as to track each person recognized by the person / group matching processing unit 103 in the captured image, and the person is included in the captured image of the network camera 100. Each group recognized by the group matching processing unit 103 is always included. In this control, the angle of view and the posture may be controlled based on the control of the server device 400.

<Step S6>
If the persons who succeeded in recognition in step S2 (here, persons A, B, and C that do not belong to the same group) have disappeared (out of frame) from the captured image, the person / group matching processing unit 103 transmits the identification information in the personal information of each of the persons A, B, and C read and used from the person / group collation data storage unit 104 until then to the server apparatus 400. The server apparatus 400 sends the person information corresponding to the identification information, that is, the person information of each of the persons A, B, and C to a network camera other than the network camera 100 (the network camera 200 in this embodiment). To send. Since the other network camera registers the personal information of each of the persons A, B, and C transmitted from the server device 400 in its own person / group collation data storage unit 104, the other network camera thereafter Recognition processing is performed using the person information registered in the person / group verification data storage unit 104, that is, the person information of each of the persons A, B, and C, and the persons A, B, and C are detected from the captured image. Can be recognized.

  For example, if a person A, B, C is recognized from the image captured by the network camera 100 at a certain time (upper part of FIG. 5), and then the person A, B, C is out of frame from the captured image, With the above processing, the person information of each of the persons A, B, and C is registered in the person / group collation data storage unit 104 of the network camera 200. After the elapse of time), the persons A, B, and C can be recognized from the image captured by the network camera 200 (lower part of FIG. 5).

<Step S7>
The person / group collation processing unit 103 uses the group information stored in the person / group collation data storage unit 104 to identify a group of persons belonging to one group among each person recognized in step S2.

  For example, identification information included in group information among identification information of each recognized person, that is, identification information of a person belonging to one group is specified, and a captured image between persons of the specified identification information Conditions such as the distance being equal to or less than the reference distance included in the group information, or the total number of persons or gender ratio of the identified identification information matching the total number or gender ratio included in the group information are satisfied. If it is determined that the group has been recognized from the captured image.

  As a result of such processing, when a group of persons belonging to one group can be identified from the captured image, the group of persons is set as the tracking target, and the process proceeds to step S8 and cannot be identified. If so, the process returns to step S2.

<Step S8>
In this group tracking, the position where the group exists is determined based on the position of the person belonging to the group.

  In this group tracking, even if a person whose threshold value is greater than or equal to the threshold value in step S8 is detected, a tracking process for the number of persons having the threshold value in step S8 is performed. Therefore, it is possible to reduce the load of processing for searching and collating for tracking. In addition, during tracking, even if a person belonging to the group overlaps and a part of the person belonging to the group cannot be detected, the group is tracked by tracking the detected person.

  The video encoding unit 105 generates a compression-encoded image by performing compression encoding on the captured image that is the processing target in step S2. The video encoding unit 105 transmits the generated compressed code image and information (analysis information) related to the tracking target to the server apparatus 400 via the LAN I / F unit 106.

  The analysis information includes, for example, identification information included in the group information among identification information of each recognized person (that is, identification information of each person included in the person group as the tracking target), tracking in the captured image The position of the target (the position of the area of each person included in the person group as the tracking target) is included. However, any information may be adopted as analysis information as long as it is information for specifying each person included in the person group as the tracking target in the captured image.

  Each time the server apparatus 400 receives a compressed code image and analysis information from the network camera 100, the server apparatus 400 stores the set of the compressed code image and analysis information in the storage device 300. As a result, for each network camera, a set group of compressed code images and analysis information transmitted from the network camera is registered in the storage device 300.

  In addition, the network camera 100 controls the angle of view and the posture of the network camera 100 so as to track each person included in the group of persons as the tracking target, and displays “as a tracking target” in the captured image of the network camera 100. "Each person included in the group of persons" is always included. In this control, the angle of view and the posture may be controlled based on the control of the server device 400.

<Step S9>
When the group of persons belonging to the group corresponding to the group information (here, persons A, B, and C belonging to the same group) disappears (frames out) from the captured image, the person / group matching process The unit 103 transmits the identification information of the group information read and used from the person / group collation data storage unit 104 until then to the server device 400. The server apparatus 400 transmits group information corresponding to the identification information to a network camera other than the network camera 100 (in the present embodiment, the network camera 200). Since the other network camera registers the group information transmitted from the server device 400 in its own person / group collation data storage unit 104, the other network camera thereafter registers in its own person / group collation data storage unit 104. Group recognition processing is performed using the group information that has been set, that is, the group information of the group of persons A, B, and C, and the group of persons A, B, and C is recognized from the captured image. can do.

  For example, if a group of persons A, B, and C is recognized from an image captured by the network camera 100 at a certain time (upper part of FIG. 6), and then this group is framed out from the captured image, With this processing, the group information of the group consisting of the persons A, B, and C is registered in the person / group collation data storage unit 104 of the network camera 200. After the elapse of time), the group consisting of persons A, B, and C can be recognized from the image captured by the network camera 200 (lower part of FIG. 6).

<Step S10>
When the personal information / group information is transmitted from the server device 400 for the above-described reason, the control unit 199 receives the personal information / group information via the LAN I / F unit 106 and receives the personal / group verification data. The person information / group information already registered in the storage unit 104 is deleted, and the received person information / group information is newly registered in the person / group verification data storage unit 104.

  In step S6 and step S9, the person information / group information held by the person / group collation data storage unit 104 of the network camera 100 becomes unnecessary, and may be deleted.

  Further, each functional unit configuring the network camera 100 illustrated in FIG. 1 may be configured using hardware. However, the control unit 199 is configured by a processor such as a CPU, the person / group verification data storage unit 104 is configured by a memory device such as a hard disk, the LAN I / F unit 106 is configured by an interface device, and other functional units are implemented by software. It doesn't matter. In this case, each process described as being performed by the network camera 100 can be realized by a processor such as a CPU executing such software. Of course, what part is configured by software and which part is configured by hardware may be appropriately designed.

  Thus, it can be seen that the network camera according to the present embodiment is merely an example of the following tracking device. That is, if the number of subjects recognized from the captured image is equal to or greater than the threshold, the subject group belonging to one group among the subject groups recognized from the captured image is set as the tracking target, and if the number is less than the threshold, the imaging This is a tracking device that targets each subject recognized from an image as a tracking target. Further, the network camera is an imaging device, and is an imaging device that transmits information on the tracking target to the control device of the imaging device. In such a case, when receiving information on the tracking target from the target imaging device, the control device transmits information for recognizing the tracking target to an imaging device other than the target imaging device.

[Second Embodiment]
As described in the first embodiment, the captured image of each frame by the network camera 100 and the captured image of each frame by the network camera 200 are displayed on the display screen of the display device 500, and on each captured image, The video analysis result for the captured image may be displayed superimposed.

  In that case, the person / group matching processing unit 103 further includes flag information indicating whether the tracking target is an individual person or a group in the analysis information described in the first embodiment, and The analysis information and the compressed code image are transmitted to the server apparatus 400.

  The display device 500 reads and decodes the compressed code image stored in the storage device 300, displays the decoded image, and uses the analysis information stored as a set with the read compressed code image. Read the flag information. If the flag information indicates that an individual person is a tracking target, the position of the individual person is read from the analysis information, and a rectangle is displayed at the read position. A rectangle can be superimposed and displayed on the person to be tracked. On the other hand, when the flag information indicates that the group is a tracking target, the position of each person belonging to the group is read from the analysis information, and a rectangle including each of the read positions is displayed. Thus, a rectangle can be superimposed and displayed on the group to be tracked on the captured image.

[Third Embodiment]
In the first embodiment, the person / group verification data storage unit 104 is provided in each network camera. However, if the bandwidth capacity of the network infrastructure of the network 600 is large, each network camera does not have the person / group matching data storage unit 104, but the server device 400 side has a person / group matching corresponding to each network camera. The data storage unit 104 may be provided. In that case, the “process between the person / group collation data storage unit 104 on the network camera side and the server device 400” described in the first embodiment is “the person / group collation data storage unit 104 of the server device 400”. Between the server device 400 and a control unit (not shown) of the server device 400 ”, and the content of the processing is the same as in the first embodiment.

  In addition, the processing performed in the server apparatus 400 configured as described above may be realized in a cluster format configuration in units of several units. In this case, a system in which the server apparatus 400 integrates the processing results in each cluster. Can also be built.

(Other examples)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

  103: Person / group verification processing unit 104: Person / group verification data storage unit

Claims (11)

A tracking device,
If the number of subjects recognized from the captured image is equal to or greater than the threshold, the subject group belonging to one group among the subject groups recognized from the captured image is set as the tracking target, and if the number is less than the threshold, the captured image A tracking device comprising means for tracking each recognized subject. Furthermore,
The tracking device according to claim 1, further comprising means for controlling an attitude or an angle of view of the tracking device so that the tracking target is contained in the captured image. A plurality of imaging devices functioning as the tracking device according to claim 1 or 2,
And a control device that controls the plurality of imaging devices.   The control device, when receiving information on the tracking target from the target imaging device, transmits information for recognizing the tracking target to an imaging device other than the target imaging device. The system described in.   5. The system according to claim 4, wherein, when the tracking target is a subject group belonging to the one group, the information regarding the tracking target includes identification information of each subject included in the subject group.   5. The system according to claim 4, wherein when the tracking target is each subject recognized from the captured image, the information regarding the tracking target includes identification information of the subject. A control method for a tracking device,
If the number of subjects recognized from the captured image is equal to or greater than the threshold, the subject group belonging to one group among the subject groups recognized from the captured image is set as the tracking target, and if the number is less than the threshold, the captured image A control method for a tracking device, characterized in that each recognized subject is set as a tracking target. Furthermore,
The method for controlling a tracking device according to claim 7, wherein information on the tracking target is transmitted to a control device of the tracking device.   9. The tracking device according to claim 8, wherein when the tracking target is a subject group belonging to the one group, the information regarding the tracking target includes identification information of each subject included in the subject group. Control method.   9. The tracking device control method according to claim 8, wherein when the tracking target is each subject recognized from the captured image, the information regarding the tracking target includes identification information of the subject.   A computer program for causing a computer to function as the tracking device according to claim 1.

Images