JP2018055521A - Detector, detection method and detection system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique of enabling detection of a mobile object whose shape is known with higher accuracy.SOLUTION: The detector includes a moving area detection unit that detects a moving area from an image and obtains moving-area detection result data, an evaluation unit that evaluates a degree of matching between a group of moving pixels constituting the moving area and shape data prepared in advance for each of a plurality of areas in the moving-area detection result data, and a determination unit that determines whether an area in which the degree of matching exceeds a threshold exists in the plurality of areas.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a detection device, a detection method, and a detection system.

  Conventionally, various techniques have been disclosed as techniques for detecting a moving object. For example, an area where a moving object exists (hereinafter also referred to as a “moving object area”) is detected from an image captured by an ITV (Industrial TeleVision) camera, and an object tracking template is created based on the detected moving object area. A technique for tracking a moving object using the created object tracking template is disclosed (for example, see Patent Document 1).

JP 2006-012013 A

  However, it is desirable to provide a technique that can detect a moving object having a known shape with higher accuracy.

  In order to solve the above problem, according to one aspect of the present invention, a moving area detection unit that detects a moving area from video and obtains moving area detection result data, and each of a plurality of areas in the moving area detection result data An evaluation unit that evaluates the degree of fit between a moving pixel group that constitutes the moving area and shape data prepared in advance, and whether or not there is an area in which the degree of fit exceeds a threshold value in the plurality of areas. And a determination unit for determining.

  According to another aspect of the present invention, a moving region is detected from an image to obtain moving region detection result data, and a moving pixel that constitutes the moving region for each of a plurality of regions in the moving region detection result data Detecting the degree of fit between the group and the shape data prepared in advance, and determining whether or not there is a region where the degree of fit exceeds a threshold value in the plurality of regions. A method is provided.

  According to another aspect of the present invention, there is provided a detection system including a camera that captures an image and a detection device, wherein the detection device detects a movement region from the image and obtains movement region detection result data. An area detection unit, an evaluation unit for evaluating the degree of fit between a moving pixel group constituting the moving region and shape data prepared in advance for each of the plurality of regions in the moving region detection result data, and among the plurality of regions And a determination unit that determines whether or not there is a region in which the degree of fit exceeds a threshold value.

  As described above, according to the present invention, a moving object having a known shape can be detected with higher accuracy.

It is a figure which shows the example of the image contained in an image | video, and the example of the movement area | region detected from the image | video. It is a figure which shows the example of the image contained in an image | video, and the example of the movement area | region detected from the image | video. It is a figure which shows the example of the image contained in an image | video, and the example of the movement area | region detected from the image | video. It is explanatory drawing which showed the structural example of the detection system which concerns on the 1st Embodiment of this invention. It is a figure which shows the example of the image imaged with a camera. It is a block diagram which shows the function structural example of the moving object area | region detection apparatus which concerns on the same embodiment. It is a flowchart which shows the operation example of the moving object area | region detection apparatus which concerns on the same embodiment. It is a figure which shows the example of the movement area | region detected by the movement area | region detection part. It is a figure which shows the example of shape data. It is a figure for demonstrating the modification of evaluation of a fit degree. It is a figure which shows the case where the appearance model which maximized the fit degree was superimposed on each of the two coordinate positions where the maximum fit degree exceeds the threshold value. It is a figure which shows the modification of an appearance model area | region. It is a figure for demonstrating the 1st method for selecting shape data. It is a figure for demonstrating the 2nd method for selecting shape data. It is a figure for demonstrating the 3rd method for selecting shape data. It is a block diagram which shows the function structural example of the moving object area | region detection apparatus which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows the operation example of the moving object area | region detection apparatus which concerns on the same embodiment. It is a figure which shows the example of the optical flow detected when a camera has a motion.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

  In the present specification and drawings, a plurality of constituent elements having substantially the same or similar functional configuration are distinguished by attaching different numerals to the same reference numerals. Further, similar constituent elements of different embodiments are distinguished by attaching different alphabets after the same reference numerals. However, when there is no need to particularly distinguish each of a plurality of constituent elements having substantially the same functional configuration, only the same reference numerals are given.

(0. Background)
First, the background of the embodiment of the present invention will be described. Various techniques have been disclosed as techniques for detecting a moving object. For example, a technique for detecting a moving object region from an image captured by an ITV camera, creating an object tracking template based on the detected moving object region, and tracking the moving object using the created object tracking template is disclosed. Yes. In such a technique, when a moving object area is detected, an area in which movement has occurred between a plurality of frames (hereinafter also referred to as “moving area”) is detected by a background difference method or an optical flow.

  However, depending on the background difference method or the optical flow, a situation may occur in which the moving region is not detected with high accuracy. This situation will be described in detail. 1A to 1C are diagrams illustrating examples of images included in a video and examples of moving areas detected from the video.

  Referring to FIG. 1A, an image G10-1 captured by a camera is shown. The image G10-1 shows a passenger car that moves to the left as the moving object G11-1. Further, referring to FIG. 1A, in order to detect a region (moving object region) where the moving object G11-1 exists, the moving region G21-1 detected by the background subtraction method or the optical flow from the video imaged by the camera. (Movement area detection result data G20-1 including movement area G21-1 at the corresponding position) is shown.

  Referring to FIG. 1B, an image G10-2 captured by the camera is shown. The image G10-2 shows a track that moves to the left as the moving object G11-2. Referring to FIG. 1B, in order to detect a region (moving object region) where the moving object G11-2 exists, the moving region G21-2 detected by the background subtraction method or the optical flow from the video imaged by the camera. (Moving area detection result data G20-2 including the moving area G21-1 at the corresponding position) is shown.

  Also, referring to FIG. 1C, an image G10-3 captured by the camera is shown. The image G10-3 shows an aircraft that moves to the left as the moving object G11-3. Further, referring to FIG. 1C, in order to detect a region (moving object region) where the moving object G11-3 exists, the moving region G21-3 detected by the background subtraction method or the optical flow from the video imaged by the camera. (Movement area detection result data G20-3 including the movement area G21-1 at the corresponding position) is shown.

  For example, referring to FIG. 1A, in the image G10-1, since the entire moving object G11-1 is moving leftward, the entire area where the moving object G11-1 exists is detected as the moving area. Is ideal. However, in practice, since the change in the image in the central area of the moving object G11-1 is small in the area where the moving object G11-1 exists, the central area of the moving object G11-1 is set as the moving area G21-1. There may be a situation in which it is not detected (the moving region G21-1 is separated in the moving direction of the moving object G11-1).

  A similar situation can occur in the examples shown in FIGS. 1B and 1C. As described above, when the moving area is detected from the video imaged by the camera by the background subtraction method or the optical flow, and the detected moving area is treated as the moving object area, the detection accuracy of the moving object area may not be improved. . If the detection accuracy of the moving object area does not improve, even if a predetermined process (for example, storage or high-quality transmission) is performed on the moving object area, the predetermined process is performed only on a part of the moving object. It can be lost.

  Therefore, in the embodiment of the present invention, shape data based on a two-dimensional model (hereinafter also referred to as “appearance model”) obtained by viewing a moving object from a predetermined direction is prepared in advance. A detection device (hereinafter also referred to as a “moving object region detection device”) detects a moving object from an image using shape data. According to this configuration, a moving object whose shape is known is detected with higher accuracy. Below, the technique which can detect a moving object from an image | video using shape data is mainly demonstrated.

  The background of the embodiment of the present invention has been described above.

(1. First embodiment)
First, a first embodiment of the present invention will be described.

(1-1. Configuration of detection system)
A configuration example of the detection system 1A according to the first embodiment of the present invention will be described with reference to FIG. FIG. 2 is an explanatory diagram showing a configuration example of the detection system 1A according to the first embodiment of the present invention. The configuration example shown in FIG. 2 can be similarly applied to the configuration example of the detection system 1B according to the second embodiment of the present invention described later. As shown in FIG. 2, the detection system 1A includes a camera 10 and a moving object area detection device 20A.

  The camera 10 and the moving object region detection device 20A are configured to be able to communicate with each other. In the example illustrated in FIG. 2, the camera 10 and the moving object area detection device 20 </ b> A are directly connected without a network. However, the camera 10 and the moving object region detection device 20A may be connected via a network. When the camera 10 and the moving object area detection device 20A are connected via a network, a plurality of cameras 10 may be connected to the network, or a plurality of moving object area detection devices 20A are connected. May be.

  In the present embodiment, the camera 10 is a fixed camera, and when images in the same imaging range are obtained by continuously capturing images in the same imaging range (for example, when the camera 10 is a surveillance camera). Assume mainly. However, the camera 10 is not limited to a fixed camera. For example, the camera 10 may be a variable camera that can change the imaging range. The variable camera may be a PTZ camera capable of pan / tilt / zoom. The camera 10 outputs the captured image to the moving object region detection device 20A.

  FIG. 3 is a diagram illustrating an example of an image captured by the camera 10. Referring to FIG. 3, an image G10-4 captured by the camera 10 is shown. The moving object G11-4 and the moving object G11-5 are shown in the image G10-4. In this specification, as shown in FIG. 3, the case where the camera 10 is installed in an airfield and the moving object is an aircraft will be mainly described. However, the place where the camera 10 is installed may be a road on which a vehicle travels, a route through which a ship passes, a train line, or the like (the moving object may be a vehicle, a ship, a train, or the like).

  The moving object region detection device 20A detects whether or not a moving object is present from an image captured by the camera 10 using shape data. When the moving object region detection device 20A detects a region where the moving object exists (moving object region), at least one of the information related to the moving object region and the information related to the shape data is not shown. (For example, a recording device (not shown), camera 10, etc.). The other device records the information output from the moving object region detection device 20A.

  The configuration example of the detection system 1A according to the first embodiment of the present invention has been described above.

(1-2. Functional Configuration of Moving Object Area Detection Device)
Subsequently, a functional configuration example of the moving object region detection device 20A according to the first embodiment of the present invention will be described. FIG. 4 is a block diagram illustrating a functional configuration example of the moving object region detection device 20A according to the first embodiment of the present invention. As illustrated in FIG. 4, the moving object region detection device 20A according to the first embodiment of the present invention includes a control unit 210A, a communication unit 230, and a storage unit 240.

  The control unit 210A has a function of controlling the entire operation of the moving object region detection device 20A, and may be configured by dedicated hardware, or a CPU built in the moving object region detection device 20A is stored in the ROM. The program may be realized by developing the program in a RAM and executing the program. In addition to providing such a program, a storage medium storing such a program may also be provided. The control unit 210A includes a moving area detection unit 211A, an evaluation unit 212, a determination unit 213, and an output control unit 214.

  The communication unit 230 is a communication interface for transmitting and receiving various information to and from the camera 10. For example, the communication unit 220 can receive video from the camera 10. Further, for example, the communication unit 230 sends at least one of information related to the moving object region and information related to the shape data to another device (not shown) (eg, a recording device not shown) or the camera 10). You may send it.

  The storage unit 240 can store a program and data for operating the control unit 210A. In addition, the storage unit 240 can temporarily store various data necessary in the course of the operation of the control unit 210A. In the first embodiment of the present invention, the storage unit 240 can store shape data.

(1-3. Detailed Function of Moving Object Area Detection Device)
Subsequently, functional details of the moving object region detection device 20A according to the first embodiment of the present invention will be described. FIG. 5 is a flowchart showing an operation example of the moving object region detection device 20A according to the first embodiment of the present invention. As shown in FIG. 5, in the moving object region detection device 20A according to the first embodiment of the present invention, the control unit 210A receives the latest video frame (image) captured by the camera 10 by the communication unit 230. It is determined whether it has been done (S11).

  When the latest video frame (image) captured by the camera 10 is not received by the communication unit 230 (“No” in S11), the control unit 210A ends the operation. On the other hand, when the latest video frame (image) captured by the camera 10 is received by the communication unit 230 (“Yes” in S11), the moving area detection unit 211A detects the moving area from the video (S12). For example, the moving area detection unit 211A detects the moving area based on the background difference method or the optical flow.

  FIG. 6 is a diagram illustrating an example of a moving area detected by the moving area detection unit 211A. As shown in FIG. 6, the moving region detection unit 211A detects the moving region G21-4 and the moving region G21-5 (moving region detection result data G20- including the moving region G21-4 and the moving region G21-5). 4 is shown). In the example shown in FIG. 6, two moving areas G21 are detected. However, the number of detected moving areas G21 is not limited to two, and may be three or more. It may be.

  Here, as in the example described with reference to FIGS. 1A to 1C, the central region of the moving object G11-4 (FIG. 3) is not detected as the moving region G21-4 (the moving region G21-4 is the moving object G11). -4 (FIG. 3). Similarly, the central area of the moving object G11-5 (FIG. 3) is not detected as the moving area G21-5 (the moving area G21-5 is separated in the moving direction of the moving object G11-5 (FIG. 3)).

  The movement area detection unit 211A generates movement area detection result data so that the movement area G21-4 and the movement area G21-5 are distinguished from other areas. In the following description, in the moving area detection result data G20-4, “1” is set for each pixel in the moving area G21-4 and the moving area G21-5, and “0” is set for each pixel in the other areas. Mainly assumed to be set. However, the pixel values set in the moving area G21-4 and the moving area G21-5 may be other than “1”.

  Subsequently, the evaluation unit 212 includes a pixel group constituting the moving region G21-4 and the moving region G21-5 (hereinafter, also referred to as “moving pixel group”) for each of a plurality of regions in the moving region detection result data G20-4. And the degree of fit between the shape data.

  FIG. 7 is a diagram illustrating an example of shape data. Referring to FIG. 7, shape data H10-1 to H10-5 are shown. In this specification, it is assumed that the shape data H10-1 to H10-5 are prepared in advance and stored in the storage unit 240. However, the number of shape data is not limited to five types, and may be one type or a plurality of types other than the five types.

  In the example shown in FIG. 7, the shape data H <b> 10-1 is a first area (hereinafter referred to as “appearance model area”) corresponding to the entire two-dimensional model of a moving object (for example, an aircraft) viewed from a predetermined direction. H11-1 and a second region other than the first region H11-1 (hereinafter, also referred to as “background region”) are generated so as to be distinguished (appearance model). Different pixel values are set for each pixel in the region H11-1 and each pixel in the background region).

  In the following description, it is mainly assumed that in the shape data H10-1, “1” is set for each pixel in the appearance model region H11-1 and “0” is set for each pixel in the background region. To do. However, as will be described later, the pixel value set in the appearance model region H11-1 may be other than “1”. Similarly, the shape data H10-2 to H10-5 are generated so that the appearance model regions H11-2 to H11-5 and the background regions are distinguished (different pixel values are set). .

  The shape data H10-1 to H10-5 may be manually generated or automatically generated by a computer. For example, when the shape data H10-1 to H10-5 are manually generated, the data generator displays the rough shape of the moving object (for example, an aircraft) viewed from each direction as the appearance model regions H11-1 to H11. What is necessary is just to draw as outline of -5 and generate shape data H10-1 to H10-5 based on these outlines.

  A plurality of areas used for evaluation of the degree of fit may be set in any way. In the following description, when the shape data H10-1 to H10-5 are superimposed on the moving area detection result data G20-4 while shifting one pixel at a time in the vertical direction or the horizontal direction, the shape data H10-1 to H10-1 are superimposed. Each region where H10-5 overlaps the moving region detection result data G20-4 is used for the evaluation of the degree of fit. However, the shift amount of the shape data H10-1 to H10-5 is not limited to one pixel, but may be a plurality of pixels.

  At this time, the evaluation unit 212 includes the pixel group (moving pixel group) and the appearance model region (for example, the moving region G21-4 and the moving region G21-5) for each of the plurality of regions in the moving region detection result data G20-4. The degree of fit may be evaluated based on the number of pixels overlapping with the appearance model region H11-1). In the following description, it is assumed that the number of overlapping pixels is treated as the degree of fit. However, as will be described below, the degree of fit is not limited to the number of overlapping pixels.

  A more specific example of the evaluation of the degree of fit will be described. In the following description, T (x, y) indicates a set of shape data that can be used at an arbitrary coordinate position (x, y) in the moving region detection result data G20-4. Here, shape data H10-1 to H10-5 are assumed as a set of shape data. Tn indicates the nth shape data extracted from the shape data set T (x, y).

  Tn (i, j) is set to “1” if an arbitrary coordinate position (i, j) in the nth shape data is inside the appearance model area. For example, when the nth shape data is the shape data H10-1, an arbitrary coordinate position (i, j) in the shape data H10-1 can be seen in Tn (i, j). Is set to “1”. On the other hand, Tn (i, j) is set to “0” if an arbitrary coordinate position (i, j) in the nth shape data is outside the appearance model region.

  D (x + i, y + j) is set to “1” if the coordinate position (x + i, y + j) in the movement area detection result data G20-4 is inside the movement area G21-4 or movement area G21-5. . On the other hand, D (x + i, y + j) is set to “0” if the coordinate position (x + i, y + j) is outside the moving region G21-4 or the moving region G21-5. Here, the inside of the moving area is not necessarily the detected moving area itself, but may be inside the range surrounded by the moving area.

  At this time, when the degree of fit at an arbitrary coordinate position (x, y) in the moving region detection result data G20-4 is evaluated for all the shape data extracted from the shape data set T (x, y), The maximum value S (x, y) of the degree of fit is expressed by the following (Formula 1).

・・・（数式１）

... (Formula 1)

  In (Formula 1), the pixel group (moving pixel group) and the appearance model region (moving pixel group) constituting the moving region G21-4 and the moving region G21-5 for each of a plurality of regions in the moving region detection result data G20-4. For example, it is assumed that the number of pixels that overlap with the appearance model region H11-1) is treated as the degree of fit. However, as described above, the degree of fit is not limited to the number of overlapping pixels. A specific example will be described.

  FIG. 8 is a diagram for explaining a modified example of the evaluation of the degree of fit. Referring to FIG. 8, the movement area detection result data G20-4 includes a movement area G21-4 and a movement area G21-5. Here, when the ground is shown in the video, it is considered that the moving object is shown large because the lower side of the ground shown in the video is closer to the camera. Therefore, as shown in FIG. 8, the shape data may be moved downward while being enlarged (the appearance model region H11-1, the appearance model region H11-1A, and the appearance model region H11-1B). .

  In such a case, since the size of the shape data itself changes, the larger the shape data, the larger the number of overlapping pixels itself. Therefore, in such a case, the ratio of the number of overlapping pixels to the total number of pixels in the shape data may be treated as the degree of fit. Alternatively, the number of pixels constituting the shape data is not necessarily the same in all the shape data. Even in such a case, the ratio of the number of overlapping pixels to the total number of pixels in the shape data may be treated as the degree of fit.

  Returning to FIG. The determination unit 213 determines whether or not there is a region whose degree of fit exceeds a threshold value among the plurality of regions in the moving region detection result data G20-4. In other words, the determination unit 213 searches for an area where the degree of fit exceeds the threshold as a place where the appearance model fits (S13). For example, the determination unit 213 determines whether there is a coordinate position (x, y) where the maximum value S (x, y) of the degree of fit exceeds a threshold value. Here, it is assumed that it is determined that there are two coordinate positions (x, y) where S (x, y) exceeds the threshold value so as to correspond to each of the movement region G21-4 and the movement region G21-5. .

  FIG. 9 is a diagram illustrating a case where the appearance model with the maximum degree of fit is superimposed on each of two coordinate positions where the maximum value S (x, y) of the degree of fit exceeds the threshold value. Referring to FIG. 9, the movement area detection result data G20-4 includes a movement area G21-4 and a movement area G21-5. An appearance model region H11-1A is superimposed on the movement region G21-4. Further, the appearance model region H11-5C is superimposed on the movement region G21-5.

  Returning to FIG. The output control unit 214 controls the output of information regarding the moving object region when there is a region (moving object region) whose degree of fit exceeds the threshold (S14). And the control part 210 transfers to S11. Here, the information regarding the moving object region is not particularly limited, but may include coordinates of a predetermined position (for example, a center position) of the moving object region. Further, the information related to the moving object area may include the size of the moving object area (for example, the vertical and horizontal lengths of the moving object area).

  Alternatively, the output control unit 214 may control the output of information related to shape data whose degree of fit with respect to the moving object region exceeds a threshold value. Information related to shape data is not particularly limited, but may include shape data itself. Alternatively, the information on the shape data may include the size of the shape data (for example, the vertical and horizontal lengths of the rectangular area of the shape data). Alternatively, the information related to the shape data may include the outline of the appearance model included in the shape data.

  When the output control unit 214 controls the output of at least one of the information related to the moving object region and the information related to the shape data (hereinafter also referred to as “output information”), the output information is not illustrated. The data is output to another device (for example, a recording device (not shown), the camera 10, etc.). The other device records output information. The output information recorded in the other device can be read and used later.

  In the above, an example in which the shape data H10-1 has the appearance model region (first region) H11-1 corresponding to the entire two-dimensional model of the moving object viewed from a predetermined direction has been shown (for example, FIG. 7). However, the entire two-dimensional model of the moving object viewed from a predetermined direction may not be used. That is, the shape data H10-1 may have an area corresponding to a part of the two-dimensional model when the moving object is viewed from a predetermined direction as the appearance model area (first area). Such an example will be specifically described.

  FIG. 10 is a diagram illustrating a modified example of the appearance model region. Referring to FIG. 10, shape data H10-6 is shown. Further, the shape data H10-6 includes a region corresponding to a part of a two-dimensional model when a moving object (for example, an aircraft) is viewed from a predetermined direction as a view model region H11-6. The evaluation unit 212 may evaluate the degree of fit based on the number of pixels overlapping between the moving pixel group and the appearance model region H11-6. For the evaluation of the degree of fit, the evaluation described above can be applied.

  It should be noted that a value (pixel value) may be set for each pixel in the appearance model region H11-6. Then, the evaluation unit 212 may evaluate the degree of fit based on the total value of the values (pixel values) of the pixels that overlap between the moving pixel group and the appearance model region H11-6. For example, the total value itself may be treated as the degree of fit, or the ratio of the total value to the total number of pixels of the shape data may be treated as the degree of fit. In this way, if a value is set for each pixel in the appearance model region H11-6, the moving object region can be detected with higher accuracy.

  Referring to FIG. 10, the appearance model region H11-6 includes a part P1 and a part P2. The site P2 is an end region in the moving direction of the moving object, and the site P1 is an inner region adjacent to the site P2. At this time, a higher value (for example, “2”, etc.) is set for each pixel in the part P2, and a next higher value (for example, “1”, etc.) is set for each pixel in the part P1, “0” may be set for each pixel in other regions (part P0 and background region). Thus, a higher value may be set for pixels closer to the end in the movement direction that are easily detected as the movement region.

  In the above, the evaluation unit 212 evaluates the degree of fit for a plurality of shape data, evaluates the maximum value thereof, and the determination unit 213 determines whether there is a region where the maximum value exceeds the threshold value. Described as an example. However, it may be assumed that the amount of processing required to evaluate the degree of fit for a plurality of shape data becomes large.

  Therefore, the evaluation unit 212 selects one shape data from among a plurality of shape data as selection data, and evaluates the degree of fit between the moving pixel group and the selection data for each of the plurality of regions in the moving region detection result data. Also good. As a method for selecting one shape data as a selection data from a plurality of shape data, several methods are assumed. The first method will be described with reference to FIG.

  FIG. 11 is a diagram for explaining a first method for selecting shape data. As shown in FIG. 11, it is assumed that the appearance of the moving object at time T0 is predicted to be close to the appearance model region H11-1. In such a case, the shape data H10-1 including the appearance model region H11-1 may be associated with the time T0 in advance. When the same prediction is made, the shape data H10-4, H10-3, H10-2, and H10-5 may be associated with each of the times T1 to T4.

  And the evaluation part 212 should just select the shape data corresponding to the present time from several shape data as selection data. For example, when the time T3 is acquired as the current time, the evaluation unit 212 may select the shape data H10-2 associated with the time T3 as selection data. As described above, if the information on the change in the appearance model of the moving object with the passage of time is registered in advance, the shape data is selected on the basis of the information on the change, so that it is necessary to evaluate the degree of fit. The throughput can be suppressed.

  FIG. 12 is a diagram for explaining a second method for selecting shape data. Referring to FIG. 12, in the moving region detection result data G20-4, the range R1 includes a moving region G21-4 of the moving object to the right, and the range R2 includes the moving region G21 of the moving object to the left. -5 is included. As described above, when the direction of the moving object is determined for each region based on the traveling direction of each runway, shape data corresponding to the direction of the moving object may be associated with each region. For example, the shape data H10-1 corresponding to the moving object orientation “right” is associated with the range R1, and the shape data H10-5 corresponding to the moving object orientation “left” is associated with the range R2. It is good to have been.

  And the evaluation part 212 should just select the shape data corresponding to the area | region where the degree of fit is evaluated from several shape data as selection data. For example, the evaluation unit 212 may select the shape data H10-1 associated with the range R1 as selection data when evaluating the degree of fit of the region belonging to the range R1. In this way, if the shape data is associated with each region, the shape data is selected based on the correspondence between the region and the shape data, so that the processing amount required for evaluating the degree of fit can be suppressed.

  FIG. 13 is a diagram for explaining a third method for selecting shape data. Referring to FIG. 13, shape data H <b> 10-1 including a moving object appearance model region H <b> 11-1 is associated with the moving object direction “right”. Similarly, the shape data H10-2 including the downward-left appearance model region H11-2 is associated with the moving object orientation “downwardly left”, and the moving object orientation “downward” is downwardly directed. Shape data H10-3 including the appearance model region H11-3 is associated.

  Further, the shape data H10-4 including the right-looking appearance model region H11-4 is associated with the moving object orientation “downwardly right”, and the left-looking appearance with respect to the moving object orientation “leftward” is associated. Shape data H10-5 including the model region H11-5 is associated. Here, it is assumed that the optical flow is detected from the video by the moving region detection unit 211A. At this time, the evaluation unit 212 can detect the direction of the moving object based on the optical flow. For example, the evaluation unit 212 can detect a direction that matches or is similar to the optical flow as the direction of the moving object.

  And the evaluation part 212 should just select the shape data corresponding to the direction of a moving object as selection data from several shape data. For example, when the optical flow is “rightward”, the evaluation unit 212 detects the direction “rightward” that matches the direction “rightward” of the optical flow as the direction of the moving object. And the evaluation part 212 should just select the shape data H10-1 matched with the direction "right direction" of a moving object as selection data. In this way, if shape data is associated with each direction of the moving object, the shape data is selected based on the correspondence between the direction of the moving object and the shape data, so the process required for evaluating the degree of fit The amount can be suppressed.

(1-4. Effects)
According to the first embodiment of the present invention, the moving region detection unit 211A that detects the moving region from the video and obtains the moving region detection result data, and configures the moving region for each of the plurality of regions in the moving region detection result data. An evaluation unit 212 that evaluates the degree of fit between the moving pixel group and the shape data prepared in advance; a determination unit 213 that determines whether or not there is a region where the degree of fit exceeds a threshold value among the plurality of regions; A moving object region detection device 20A is provided.

  According to this configuration, it is possible to detect a moving object whose shape is known with higher accuracy. Therefore, when a predetermined process (for example, storage or high-quality transmission) is performed on the moving object region, an effect that the predetermined process is performed on the entire moving object is enjoyed.

  The first embodiment of the present invention has been described above.

(2. Second Embodiment)
Subsequently, a second embodiment of the present invention will be described. In the second embodiment of the present invention, it is assumed that the camera 10 has a movement for some reason. For example, the camera 10 may move due to shaking. Alternatively, when the camera 10 is a PTZ camera, the camera 10 may have a movement by a pan operation or a tilt operation.

  Here, the movement of the camera 10 may include rotation of the camera 10. Further, the movement of the camera 10 may include translation of the camera 10. For example, translation of the camera 10 can occur when the camera 10 rotates in a state where there is a deviation between the rotation axis of the camera 10 and the imaging position.

  As described above, when the camera 10 has a movement for some reason, the video can be changed not only by the movement of the moving object but also by the movement of the camera 10. Therefore, in the second embodiment of the present invention, in order to improve the detection accuracy of the moving area due to the movement of the moving object, the change due to the movement of the camera 10 is removed from the change in the image, and the change due to the movement of the camera 10 is detected. A technique for detecting a moving area based on a change in video after removal will be mainly described.

(2-1. Configuration of detection system)
First, a configuration example of a detection system 1B (FIG. 2) according to the second embodiment of the present invention will be described with reference to FIG. As shown in FIG. 2, the detection system 1B according to the second embodiment of the present invention has a configuration of the moving object region detection device 20B as compared with the detection system 1A according to the first embodiment of the present invention. This is different from the configuration of the moving object region detection device 20A. Therefore, in the second embodiment of the present invention, a configuration example of the moving object region detection device 20B will be mainly described.

(2-2. Functional Configuration of Moving Object Area Detection Device)
Next, a functional configuration example of the moving object region detection device 20B according to the second embodiment of the present invention will be described. FIG. 14 is a block diagram illustrating a functional configuration example of the moving object region detection device 20B according to the second embodiment of the present invention. As shown in FIG. 14, the moving object region detection device 20B according to the second embodiment of the present invention is compared with the moving object region detection device 20A according to the first embodiment of the present invention, as a control unit 210B. The configuration of the moving area detection unit 211B is different from the configuration of the moving area detection unit 211A of the control unit 210A. Therefore, in the second embodiment of the present invention, the configuration of the moving region detection unit 211B will be mainly described.

(2-3. Detailed functions of moving object area detection device)
Subsequently, functional details of the moving object region detection device 20B according to the second embodiment of the present invention will be described. FIG. 15 is a flowchart showing an operation example of the moving object region detection device 20B according to the second embodiment of the present invention. As shown in FIG. 15, in the moving object region detection device 20B according to the second embodiment of the present invention, the control unit 210B receives the latest video frame (image) captured by the camera 10 by the communication unit 230. It is determined whether it has been done (S11).

  When the latest video frame (image) captured by the camera 10 is not received by the communication unit 230 (“No” in S11), the control unit 210B ends the operation. On the other hand, when the latest video frame (image) captured by the camera 10 is received by the communication unit 230 (“Yes” in S11), the moving region detection unit 211B detects an optical flow from the video (S21).

  FIG. 16 is a diagram illustrating an example of an optical flow detected when the camera 10 has a motion. As shown in FIG. 16, the optical flow F <b> 10 includes not only an optical flow due to movement of a moving object but also an optical flow due to movement of the camera 10 when the camera 10 has movement. In particular, the optical flow F10 indicates an optical flow detected when the camera 10 rotates to the right and the viewpoint of the camera translates to the left. This translation occurs because the camera rotates around an axis in front of the viewpoint. However, the rotation and translation directions of the camera 10 are not particularly limited.

  As shown in FIG. 16, when the camera 10 is rotated rightward, an optical flow in the opposite direction (leftward) to the rotation of the camera 10 appears on the entire screen. However, as the viewpoint of the camera 10 translates to the left along with the rotation, an optical flow in the opposite direction (rightward) to the translation of the camera 10 appears on the entire screen. As a result, the leftward optical flow is slightly suppressed. The degree of suppression of the leftward optical flow due to translation of the camera 10 increases as the distance from the camera 10 decreases.

  That is, as shown in the optical flow F10, the lower side of the upper side of the ground where the ground appears is closer to the camera 10, so the degree of suppression of the leftward optical flow is stronger at the lower side than the upper side of the ground shown in the video. Become. Further, since the ground is closer to the camera 10 than the sky, the degree of suppression of the leftward optical flow is stronger on the ground in the video than in the sky in the video. Thus, a leftward optical flow due to the movement of the camera 10 appears on the entire screen.

  On the other hand, as shown in the optical flow F10, the moving object (corresponding to the moving area F11-1) is moving rightward. Therefore, a rightward optical flow appears at the end in the moving direction (left-right direction) of the moving object (corresponding to the moving region F11-1). As a result, the leftward optical flow due to the movement of the camera 10 is slightly suppressed at the end in the moving direction (left-right direction) of the moving object (corresponding to the moving area F11-1).

  Further, as shown in the optical flow F10, the moving object (corresponding to the moving area F11-2) is moving leftward. Therefore, a leftward optical flow appears at the end in the moving direction (left-right direction) of the moving object (corresponding to the moving region F11-2). As a result, the leftward optical flow due to the movement of the camera 10 is emphasized at the end in the moving direction (left-right direction) of the moving object (corresponding to the moving region F11-2).

  Returning to FIG. 15, the description will be continued. Subsequently, the moving region detection unit 211B generates a motion component of the camera 10 (optical flow due to the motion of the camera 10) (S22). The movement component of the camera 10 may include a component due to rotation of the camera 10 (optical flow due to rotation of the camera 10), or a component due to translation of the viewpoint accompanying rotation of the camera 10 (optical flow due to translation of the camera 10). ) May be further considered. In the following, an example will be described in which both the component due to the rotation of the camera 10 and the component due to the translation of the camera 10 are considered in the motion component of the camera 10.

  If the correspondence between the rotation per unit angle (for example, 1 degree) of the camera 10 and the optical flow generated by the rotation is measured in advance, the component due to the rotation of the camera 10 is measured by the moving region detection unit 211B. It can be generated based on the ten rotation angles and their corresponding relationships. At this time, the rotation angle of the camera 10 is obtained from a control signal of the camera 10 or the like. In the following description, the pan / tilt of the camera 10 will be described as an example of the rotation angle of the camera 10.

  Further, the component due to the translation of the viewpoint accompanying the rotation of the camera 10 is the distance from the camera 10 to the object shown in the video (that is, the angle of the ground shown in the video and the boundary position between the ground and the ground shown in the video). Can depend. Therefore, if parameters such as the angle between the direction of the camera 10 and the ground and the height of the camera 10 from the ground are determined, the moving area detection unit 211B corresponds to the coordinate position (x , Y), the component due to translation of the camera 10 is determined. These parameters are also obtained from the control signal of the camera 10 and the like.

  A more specific example of generating the component due to the rotation of the camera 10 and the component due to the translation of the camera 10 will be described. Now, let us say that the optical flow that occurs when an infinite point appears at the coordinate position (x, y) on the screen due to panning per unit angle φ by the camera 10 is (X (φ), Y (φ)). The optical flow that occurs when an infinite point is reflected at the coordinate position (x, y) on the screen due to the tilt per unit angle ψ by the camera 10 is defined as (X (ψ), Y (ψ)).

  The actual pan angle of the camera 10 is Φ, and the actual tilt angle of the camera 10 is ψ. Also, let D (x, y) be a component due to translation of the camera 10 at the coordinate position (x, y) on the screen. At this time, assuming that the optical flow actually generated at the coordinate position (x, y) on the screen is (X (Φ, Ψ, x, y), Y (Φ, Ψ, x, y)), (X (Φ , Ψ, x, y), Y (Φ, Ψ, x, y)) are expressed by the following (Equation 2) and (Equation 3) (the component due to the rotation of the camera 10 and the component due to the translation of the camera 10). Obtained by multiplication with).

・・・（数式２）

... (Formula 2)

・・・（数式３）

... (Formula 3)

  Subsequently, the moving region detection unit 211B removes the motion component of the camera 10 generated in this way (optical flow due to the motion of the camera 10) from the optical flow detected from the video. Then, the moving region detection unit 211B detects the moving region based on the optical flow after the component due to the camera motion is removed from the optical flow detected from the video (S12). The detection of the moving area can be performed in the same manner as the detection of the moving area in the first embodiment of the present invention. S13 and S14 can also be performed as described in the first embodiment of the present invention.

(2-4. Effect)
According to the second embodiment of the present invention, the moving region detection for detecting the moving region based on the optical flow after removing the component due to the movement of the camera 10 that captures the image from the optical flow detected from the image. A moving object region detection device 20B is provided that includes a unit 211B and includes an evaluation unit 212 and a determination unit 213, as in the first embodiment of the present invention.

  According to such a configuration, it is possible to detect a moving object having a known shape with higher accuracy, as in the first embodiment. Therefore, when a predetermined process (for example, storage or high-quality transmission) is performed on the moving object region, an effect that the predetermined process is performed on the entire moving object is enjoyed.

  Furthermore, according to such a configuration, it is possible to exclude a component due to the movement of the camera 10, improve the detection accuracy of the moving region due to the movement of the moving object, and detect the moving object with higher accuracy. As an example, even when the camera 10 is a PTZ camera and the camera 10 has a movement by a pan operation or a tilt operation, it is possible to detect a moving object with higher accuracy.

  The second embodiment of the present invention has been described above.

  The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

  For example, in the above description, the case where the output control unit 214 controls the output of at least one of the information related to the moving object region and the information related to the shape data (output information) has been described. In the above description, the output information is output to another device (not shown) (for example, a recording device (not shown), the camera 10, etc.). However, the output destination of the output information is not limited to the recording device and the camera 10 (not shown).

  For example, when the moving object region detection device 20A is connected to a computer (not shown), the output information may be transmitted to the computer. At this time, the computer may perform a predetermined display based on the received output information. For example, the computer may display a video and may perform a predetermined display (for example, display of a red rectangular frame) on the moving object area in the video based on the received information regarding the moving object area. This makes it easier for the controller to monitor moving objects, such as when a computer is installed in the control room.

  Further, for example, when the camera 10 is a PTZ camera, panning / tilting may be performed so that the moving object region is reflected at a predetermined position (for example, the center position) of the video based on the received information on the moving object region. Good. At this time, the camera 10 may perform zooming so that the moving object is enlarged and shown in the video. Then, the moving object is automatically tracked so that the moving object is captured in a position that is easy to see at a position where the image is easy to see. Alternatively, the moving object area detection device 20A may be connected to a PTZ camera different from the camera 10, and the same operation may be performed by the PTZ camera.

  Alternatively, a case where a moving region is detected and a region where the degree of fit with one or a plurality of shape data prepared in advance exceeds a threshold value is also assumed. At this time, the detected moving object may be an unknown moving object. Therefore, in such a case, the output control unit 214 may control a predetermined output (for example, predetermined visual information, predetermined audio information, etc.) for notifying the detection of an unknown moving object. In addition, since the unknown moving object may be a suspicious object or an intruder, the predetermined output for informing the detection of the unknown moving object may be a predetermined output for informing the abnormal state. .

1 (1A, 1B) Detection system 10 Camera 20 (20A, 20B) Moving object area detection device (detection device)
210 (210A, 210B) Control unit 211 (211A, 211B) Moving region detection unit 212 Evaluation unit 213 Determination unit 214 Output control unit 220 Communication unit 230 Communication unit 240 Storage unit F10 Optical flow F11 Moving region G11 Moving object G20 Moving region detection Result data G21 Moving area H10 Shape data

Claims (15)

A moving area detection unit that detects a moving area from video and obtains moving area detection result data;
An evaluation unit that evaluates the degree of fit between a moving pixel group that constitutes the moving region and shape data prepared in advance for each of a plurality of regions in the moving region detection result data;
A determination unit that determines whether or not there is a region in which the degree of fit exceeds a threshold value among the plurality of regions;
A detection device comprising: The detection device includes:
An output control unit that controls the output of at least one of the information on the region and the information on the shape data when there is a region where the degree of fit exceeds the threshold;
The detection device according to claim 1. In the shape data, a first area corresponding to all or part of a two-dimensional model of a predetermined moving object viewed from a predetermined direction and a second area other than the first area are distinguished. Generated,
The detection device according to claim 1. The evaluation unit evaluates the degree of fit based on the number of pixels overlapping between the moving pixel group and the first region;
The detection device according to claim 3. In the first area, a value is set for each pixel,
The evaluation unit evaluates the degree of fit based on a total value of values of pixels that overlap between the moving pixel group and the first region.
The detection device according to claim 4. The evaluation unit evaluates the maximum value of the degree of fit between the moving pixel group and the plurality of shape data for each of the plurality of regions.
The determination unit determines whether or not there is a region where the maximum value exceeds the threshold;
The detection device according to claim 1. The evaluation unit selects one shape data from a plurality of shape data as selection data, and evaluates the degree of fit between the moving pixel group and the selection data for each of the plurality of regions.
The detection device according to claim 1. The evaluation unit selects, as the selection data, shape data corresponding to the current time from the plurality of shape data.
The detection device according to claim 7. The evaluation unit selects, as the selection data, shape data corresponding to a range to which the plurality of regions belong from the plurality of shape data.
The detection device according to claim 7. The evaluation unit selects the selection data from the plurality of shape data based on an optical flow detected from the video.
The detection device according to claim 7. The detection device includes:
An output control unit that controls a predetermined output for notifying detection of an unknown moving object when the moving region is detected and there is no region in which the degree of fit exceeds the threshold;
The detection device according to claim 1. The moving area detection unit detects the moving area based on a background difference method or an optical flow.
The detection device according to claim 1. The moving area detection unit detects the moving area based on an optical flow after removing a component due to a movement of a camera that captures the video from an optical flow detected from the video.
The detection device according to claim 1. To detect the moving area from the video and obtain the moving area detection result data;
Evaluating a degree of fit between a moving pixel group constituting the moving area and shape data prepared in advance for each of a plurality of areas in the moving area detection result data;
Determining whether there is an area in which the degree of fit exceeds a threshold among the plurality of areas;
A detection method comprising: A detection system having a camera for capturing video and a detection device,
The detection device includes:
A moving area detection unit that detects a moving area from video and obtains moving area detection result data;
An evaluation unit that evaluates the degree of fit between a moving pixel group that constitutes the moving region and shape data prepared in advance for each of a plurality of regions in the moving region detection result data;
A determination unit that determines whether or not there is a region in which the degree of fit exceeds a threshold value among the plurality of regions;
A detection system comprising:

Images