JP2017163510A - Imaging control program, imaging control method and imaging control device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging control program capable of suppressing reduction of tracking accuracy, an imaging control method and an imaging control device.SOLUTION: An imaging control program causes a computer to execute processing for identifying a person region from an image captured by an imaging apparatus. The imaging control program causes the computer to execute, in the case where multiple person regions are identified in the captured image, processing for calculating positional relationships and moving speeds of the multiple person regions from multiple images captured by the imaging apparatus respectively. The imaging control program causes the computer to execute, based on the positional relationships in the images in the identified multiple person regions and the moving speeds of the multiple person regions, processing for controlling a frame rate of the imaging apparatus.SELECTED DRAWING: Figure 22

Description

  The present invention relates to a shooting control program, a shooting control method, and a shooting control apparatus.

  In recent years, surveillance cameras are used for various purposes such as marketing and traffic volume surveys in addition to surveillance purposes. For example, a surveillance camera is used for grasping a movement path of a person by tracking a person whose features such as clothes color are similar from a photographed image.

JP 2011-10276 A

  However, in the conventional technique, an incorrect correspondence with another person occurs, and the tracking accuracy may decrease. For example, if a person region overlaps when a person intersects, an incorrect correspondence with another person may occur, resulting in a decrease in tracking accuracy.

  In one aspect, an object is to provide an imaging control program, an imaging control method, and an imaging control apparatus that can suppress a decrease in tracking accuracy.

  In the first plan, the shooting control program causes a computer to execute processing for specifying a person area from an image shot by a shooting device. The shooting control program is a process for calculating a positional relationship and a moving speed of each of the plurality of person areas from the plurality of images shot by the shooting device when the computer specifies a plurality of person areas in the shot image. Is executed. The imaging control program causes the computer to execute processing for controlling the frame rate of the imaging device based on the positional relationship in the images of the specified plurality of person areas and the moving speed of the plurality of person areas.

  According to one embodiment of the present invention, it is possible to suppress a decrease in tracking accuracy.

FIG. 1 is a diagram illustrating an example of a schematic configuration of a system. FIG. 2 is a diagram illustrating an example of a functional configuration of the imaging control apparatus. FIG. 3 is a diagram illustrating an example of a data configuration of person tracking information. FIG. 4 is a diagram illustrating an example of similarity between person regions. FIG. 5 is a diagram illustrating an example of the movement amount of the person area. FIG. 6 is a diagram illustrating an example of the prediction of the movement position of the person area. FIG. 7 is a diagram illustrating an example of calculating the prediction accuracy. FIG. 8 is a diagram illustrating an example of calculating a prediction error. FIG. 9 is a diagram illustrating an example of the distance between the movement positions of the person area. FIG. 10 is a diagram illustrating an example of a region within a predetermined range from the edge of the image. FIG. 11 is a diagram illustrating an example of a flow for specifying a person area. FIG. 12 is a diagram illustrating an example of a flow of determining a person area based on the overlapping rate. FIG. 13 is a diagram illustrating an example of a flow for tracking a person area. FIG. 14 is a flowchart illustrating an example of a control processing procedure according to the first embodiment. FIG. 15 is a flowchart illustrating an example of a procedure of similarity calculation processing. FIG. 16 is a flowchart illustrating an example of a procedure of speed calculation processing. FIG. 17 is a flowchart illustrating an example of the procedure of the frame rate control process. FIG. 18 is a flowchart illustrating an example of a procedure of person detection processing. FIG. 19 is a flowchart illustrating an example of the procedure of the person tracking process. FIG. 20 is a diagram illustrating an example of movement of a person area. FIG. 21 is a diagram illustrating an example of movement of a person area. FIG. 22 is a diagram illustrating an example of changing the frame rate. FIG. 23 is a flowchart illustrating an example of a procedure of control processing according to the second embodiment. FIG. 24 is a flowchart illustrating an exemplary procedure of a frame rate control process according to the second embodiment. FIG. 25 is a diagram illustrating a computer that executes an imaging control program.

  Hereinafter, embodiments of a shooting control program, a shooting control method, and a shooting control apparatus according to the present invention will be described in detail with reference to the drawings. The disclosed technology is not limited by the present embodiment. Moreover, you may combine suitably the Example shown below in the range which does not cause contradiction.

[System configuration]
First, an example of a system according to the first embodiment will be described. FIG. 1 is a diagram illustrating an example of a schematic configuration of a system. The system 10 is a system that performs monitoring with a monitoring camera. Surveillance cameras are used for various purposes such as marketing and traffic surveys in addition to surveillance purposes. For example, a surveillance camera is used for grasping a movement path of a person by tracking a person whose features such as clothes color are similar from a photographed image.

  As shown in FIG. 1, the system 10 includes an imaging control device 11 and a monitoring camera 12. The imaging control device 11 and the monitoring camera 12 are connected to be communicable via a network N, and can exchange various kinds of information. As an aspect of such a network N, any type of communication network such as mobile communication such as a cellular phone, a local area network (LAN), or a virtual private network (VPN) can be adopted regardless of wired or wireless. In the example of FIG. 1, the case where three monitoring cameras 12 are provided is illustrated, but the present invention is not limited to this, and the number of monitoring cameras 12 can be an arbitrary number.

  The photographing control device 11 is a device that controls photographing of the monitoring camera 12. The imaging control device 11 is a computer such as a server computer, for example. The imaging control device 11 may be implemented as a single computer or may be implemented by a plurality of computers. In this embodiment, a case where the photographing control device 11 is a single computer will be described as an example.

  The imaging control device 11 transmits instruction information for instructing the imaging frame rate to the monitoring camera 12. In addition, the imaging control device 11 receives image data of an image captured by the monitoring camera 12 from the monitoring camera 12. The imaging control device 11 tracks a person whose characteristics such as clothes color are similar from the image of the received image data, and specifies the movement path of the person.

  The surveillance camera 12 is a photographing device capable of photographing an image. The monitoring camera 12 is arranged at a place where the monitoring target can be monitored so that the monitoring target falls within the angle of view. The surveillance camera 12 includes an image sensor such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The surveillance camera 12 can selectively capture images at a plurality of frame rates. When the monitoring camera 12 receives the instruction information from the imaging control device 11, the monitoring camera 12 captures an image at the frame rate of the instruction information. The monitoring camera 12 transmits the image data of the captured image to the imaging control device 11.

[Configuration of Shooting Control Device]
Next, the configuration of the imaging control device 11 will be described. FIG. 2 is a diagram illustrating an example of a functional configuration of the imaging control apparatus. As illustrated in FIG. 2, the imaging control device 11 includes a communication I / F (interface) unit 20, a storage unit 21, and a control unit 22.

  The communication I / F unit 20 is an interface that controls communication with other devices. As the communication I / F unit 20, a network interface card such as a LAN card can be adopted. The communication I / F unit 20 transmits and receives various types of information to and from other devices via the network N.

  The storage unit 21 is a storage device that stores various data. For example, the storage unit 21 is a storage device such as a hard disk, an SSD (Solid State Drive), or an optical disk. Note that the storage unit 21 may be a semiconductor memory that can rewrite data, such as a random access memory (RAM), a flash memory, and a non-volatile static random access memory (NVSRAM).

  The storage unit 21 stores an OS (Operating System) executed by the control unit 22 and various programs. For example, the storage unit 21 stores various programs including programs that execute various processes. Furthermore, the storage unit 21 stores various data used in programs executed by the control unit 22. For example, the storage unit 21 stores image data 30 and person tracking information 31.

  The image data 30 is data in which image data captured at a frame rate specified by the monitoring camera 12 is stored. The image data 30 may be image data of each captured image, or may be moving image data obtained by encoding each captured image using a predetermined method. The image data 30 is stored for each monitoring camera 12. The image data 30 includes information indicating the timing at which the image was captured, such as the time at which the image was captured. For example, the image data 30 includes time indicating the elapsed time from the start of shooting as the timing at which the image was shot.

  The person tracking information 31 is data storing information related to the person to be tracked. For example, the person tracking information 31 stores the identification information of the person to be tracked and the position information of the person area in each image. The person tracking information 31 is stored for each monitoring camera 12.

  In the following, in order to simplify the description, processing for the image data 30 and the person tracking information 31 stored for one monitoring camera 12 will be described. The imaging control device 11 performs similar processing on the image data 30 and the person tracking information 31 stored for each monitoring camera 12.

  FIG. 3 is a diagram illustrating an example of a data configuration of person tracking information. As shown in FIG. 3, the person tracking information 31 includes items of “person ID”, “time”, “person area”, “movement amount”, and “frame rate”. The item of person ID is an area for storing an identification number for identifying a person included in the image. For example, when it is detected that a new person is included in the image, a unique person ID (identification) is assigned to the detected person in the order of detection. The assigned person ID is stored in the person ID item. The time item is an area for storing a time indicating an elapsed time from the start of photographing as identification information of an image including the person area of the person with the person ID. In the present embodiment, the time is indicated by an integer value for the sake of simplicity. The item of the person area is an area for storing information related to the person area of the person with the person ID. The items in the person area are divided into x-coordinate items, y-coordinate items, width items, and height items. The x-coordinate item and the y-coordinate item are areas for storing information indicating the position of the person area of the person with the person ID. For example, when the position in the image is indicated by the x-coordinate and the y-coordinate, the x-coordinate of a predetermined reference position of the person area is stored in the x-coordinate item. In the y coordinate item, the y coordinate of a predetermined reference position of the person area is stored. The reference position may be, for example, the position of the center of the person area, the position of any vertex of a rectangle constituting the person area, or the position of a specific part of the person. The item of width and the item of height are areas for storing information indicating the size of the person area. For example, when the x coordinate is the width direction and the y coordinate is the height direction in the image, the number of pixels in the x coordinate direction of the person region is stored in the width item. The number of pixels in the y coordinate direction of the person area is stored in the height item. The item of movement amount is an area for storing the movement amount between images of the person area of the person with the person ID. For example, the movement amount item stores the movement amount of a predetermined reference position of the person area from the image one frame before. When there is no image one frame before, “−” indicating that there is no image one frame before is stored in the item of movement amount. The position of the person area in which “−” is stored in the item of movement amount is the position where it first appears in the image. The frame rate item is an area for storing a frame rate when an image including a person area is captured.

  In the example of FIG. 3, the person area with the person ID “1” is included in the image at time “19”, and the x coordinate “10” pixel, the y coordinate “20” pixel, the width “16” pixel, and the height. Since “60” pixels and the item of movement amount are “−”, this indicates that the position first appears in the image. Further, the image at time “19” indicates that the frame rate is “5” fps (frame per second). The person area with the person ID “1” is included in the image at time “20”, and the x coordinate is “12” pixels, the y coordinate is “27” pixels, the width is “17” pixels, and the height is “62” pixels. This indicates that the image at time “19” has moved +2 pixels in the x coordinate direction and +7 pixels in the y coordinate direction. The image at time “20” indicates that the frame rate was “5” fps.

  Returning to FIG. 2, the control unit 22 is a device that controls the entire imaging control apparatus 11. As the control unit 22, an electronic circuit such as a CPU (Central Processing Unit) or MPU (Micro Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array) can be employed.

  The control unit 22 has an internal memory for storing programs and data defining various processing procedures, and executes various processes using these. The control unit 22 functions as various processing units by operating various programs. For example, the control unit 22 includes an acquisition unit 40, a calculation unit 41, an imaging control unit 42, a specifying unit 43, and a tracking unit 44.

  The acquisition unit 40 performs various acquisitions. For example, the acquisition unit 40 acquires image data captured by the monitoring camera 12 by acquiring image data received from the monitoring camera 12 by the communication I / F unit 20.

  The calculation unit 41 performs various calculations. For example, when a plurality of person areas are specified in the captured image, the calculation unit 41 calculates the movement speed of each of the plurality of person areas. For example, the calculation unit 41 calculates the moving speeds of the plurality of person areas from the plurality of images captured by the monitoring camera 12. For example, the calculation unit 41 refers to the person tracking information 31 and determines whether two or more person regions are included in the image. For example, the calculation unit 41 determines whether two or more person areas are included in the image with the latest time of the person tracking information 31. For example, when the person tracking information 31 is in the state shown in FIG. 3 and the current time is t, the calculation unit 41 determines whether there are two or more person IDs in which “t−1” is registered in the time item. judge.

  When the captured image includes two or more person areas, the calculation unit 41 calculates the similarity between the person areas in the image. FIG. 4 is a diagram illustrating an example of similarity between person regions. In the example of FIG. 4, two person areas A and B are included in the image. For example, the calculation unit 41 calculates the similarity between the person areas A and B. Any method may be used as the similarity calculation method. An example of a similarity calculation method is normalized cross correlation (NCC). For example, the calculation unit 41 calculates the degree of similarity by calculating the normalized cross-correlation of the person areas A and B. For example, the calculation unit 41 sets one of the two person areas A and B as the template T and the other as the comparison target I. For example, the template T defines the coordinates (x ′, y ′) with the upper left coordinates of the person area as (0, 0). The height of the person area of the template T is h pixels, and the width of the person area of the template T is w pixels. The difference between the positions of the two person areas A and B is (x, y). The calculation unit 41 calculates the similarity R (x, y) for each pixel by performing the calculation shown in the following formula (1), and calculates the similarity by calculating the average of the similarity R (x, y) for each pixel. Calculate the degree. In the normalized cross-correlation, the similarity is calculated closer to 1 as the person region is more similar.

  In this embodiment, the person tracking information 31 stores information related to the person area included in each image taken by the monitoring camera 12. Therefore, when two or more person areas are included, the calculation unit 41 refers to the person tracking information 31 and obtains the person area of each person in the image. For example, when the person tracking information 31 is in the state shown in FIG. 3 and the current time is t, the calculation unit 41 calculates the person area of the person with each person ID in which “t−1” is registered in the time item. Read the position and size information. In the example of FIG. 3, information on the position and size of the person areas of the persons with the person IDs “1” and “N” is read. The calculation unit 41 uses the above formula (1) for the person areas with the person IDs “1” and “N” of the image data 30 captured at the time “t−1” stored in the storage unit 21. Perform the calculation to calculate the similarity. When the image includes three or more person areas, the calculation unit 41 calculates the similarity for each combination of two person areas.

  The calculation unit 41 determines whether there are person areas similar to each other in the person area based on the similarity between the person areas. For example, the calculation unit 41 determines whether there is a calculated similarity between person areas that is equal to or greater than a predetermined threshold. This threshold value is set to a value that the person areas are considered to be similar to. This threshold value may be a fixed value or may be adjustable from the outside by a person using the imaging control device 11.

  When there is a similar person area in the image, the calculation unit 41 calculates the moving speed of each person area included in the image. The calculation unit 41 calculates the moving speed of the person area from the plurality of image data 30 stored in the storage unit 21. For example, the calculation unit 41 calculates the movement speed of the person area from the images of the two image data 30 that have been captured most recently stored in the storage unit 21. For example, the calculation unit 41 obtains the movement amount of the person area for each person area from two past images. FIG. 5 is a diagram illustrating an example of the movement amount of the person area. In the example of FIG. 5, in the person region with the person ID “N”, the position at time t−2 is (px (t−2), py (t−2)), and the position at time t−1 is (px (T-1), py (t-1)). In this case, the calculation unit 41 calculates the movement amount (Δx, Δy) of the person area with the person ID “N” from the following equations (2) and (3).

Δx = px (t−1) −px (t−2) (2)
Δy = py (t−1) −py (t−2) (3)

  Here, in this embodiment, the person tracking information 31 stores information related to the person area included in each image captured by the monitoring camera 12. Therefore, the calculation unit 41 refers to the person tracking information 31 and obtains the movement amount of the person area for each person area. For example, when the person tracking information 31 is in the state illustrated in FIG. 3, the calculation unit 41 reads the movement amount (1, 3) of the person ID “N” in which “t−1” is registered in the time item.

  The calculation unit 41 obtains a movement speed from the movement amount of the person area for each person area. In this embodiment, the calculation unit 41 obtains the movement amount of the person area at 1 fps as the movement speed by multiplying the movement amount (Δx, Δy) of the person area by the frame rate fp at the time of shooting. For example, the calculation unit 41 calculates the movement speed (dfx, dfy) per second of the person area from the following equations (4) and (5).

dfx = Δx × fp (4)
dfy = Δy × fp (5)

  The imaging control unit 42 controls various types of imaging. The imaging control unit 42 can selectively designate the frame rate of the monitoring camera 12 from a plurality of predetermined frame rates. When the plurality of frame rates are used for tracking a person, the tracking accuracy satisfies a predetermined standard. The imaging control unit 42 transmits instruction information for instructing the frame rate of imaging to the monitoring camera 12 to control the frame rate of the monitoring camera 12.

  Here, for example, when the frame rate is high, the surveillance camera 12 can obtain an image with little movement of a person. For this reason, in order to track a person with high accuracy, a higher frame rate is preferable. However, when the frame rate increases, the monitoring camera 12 shortens the imaging cycle and transmits a lot of image data. When the frame rate of the monitoring camera 12 increases, the imaging control device 11 receives more image data and increases the processing load. In particular, as shown in FIG. 1, when the imaging control apparatus 11 receives image data from a plurality of monitoring cameras 12, the imaging control apparatus 11 increases from each monitoring camera 12 when the frame rate of each monitoring camera 12 increases. Since image data is received, the processing load is very high. In order to suppress the processing load, it is preferable that the monitoring camera 12 has a low frame rate. However, when the frame rate is low, the monitoring camera 12 may not be able to track the person because the amount of movement of the person between the captured images increases. As described above, when tracking a person, the tracking accuracy and the processing load amount are in a trade-off relationship with respect to the frame rate. For this reason, the imaging control device 11 controls the processing load by setting the frame rate of the monitoring camera 12 to the lowest frame rate among a plurality of frame rates whose tracking accuracy satisfies a predetermined standard during normal times. . For example, the imaging control unit 42 can selectively designate the frame rate of the monitoring camera 12 from 30 fps, 15 fps, and 5 fps. In this case, the imaging control device 11 controls the frame rate of the monitoring camera 12 to 5 fps during normal times.

  By the way, when the imaging control apparatus 11 tracks a person from an image captured by the monitoring camera 12 and obtains a movement route, if there are a plurality of person areas in the image, an erroneous correspondence with another person occurs. Tracking accuracy may be reduced. For example, if a person region overlaps when a person intersects, an incorrect correspondence with another person may occur, resulting in a decrease in tracking accuracy. When a plurality of person regions included in an image are similar to each other, an erroneous correspondence with another person is particularly likely to occur.

  Therefore, the shooting control unit 42 controls the frame rate of the monitoring camera 12 when a plurality of person areas are specified in the shot image and there are person areas similar to each other in the plurality of person areas. For example, the imaging control unit 42 predicts the movement position at the next imaging timing for each of a plurality of selectable frame rates for a plurality of human regions of a captured image. Then, the imaging control unit 42 determines the frame rate at which the distance between the predicted movement positions is larger than the prediction error as the frame rate of the monitoring camera 12. The imaging control unit 42 transmits instruction information indicating the determined frame rate to the monitoring camera 12 to control the frame rate of the monitoring camera 12.

  On the other hand, when a plurality of person areas are not specified in the captured image or when there are no person areas similar to each other in the plurality of person areas, the shooting control unit 42 is the lowest of a plurality of selectable frame rates. The frame rate is determined as the frame rate of the monitoring camera 12. The imaging control unit 42 transmits instruction information indicating the determined frame rate to the monitoring camera 12 to control the frame rate of the monitoring camera 12.

  Here, prediction of the movement position of the person area will be described. For example, the position of the person area at time t−1 is (x (t−1), y (t−1)), the frame rate is f, and the moving speed of the person area per second is (dfx, dfy). ). In this case, the movement position (x, y) in the next photographing can be obtained from the following equations (6) and (7).

x = dfx × (1 / f) + x (t−1) (6)
y = dfy × (1 / f) + y (t−1) (7)

  FIG. 6 is a diagram illustrating an example of the prediction of the movement position of the person area. The movement speed (dfx, dfy) is calculated from the movement amount of the person area at time t-2 and time t-1. The moving position (x, y) at time t is a period of 1 / f at the moving speed (dfx, dfy) from the position (x (t-1), y (t-1)) of the person area at time t-1. The position has been moved. Since the movement position (x, y) is a predicted position, a prediction error ε may occur unlike the actual position of the human region at time t. That is, the movement position (x, y) includes the prediction error ε.

The prediction error can be calculated from the prediction accuracy. The prediction accuracy can be obtained according to the prediction method. For example, the prediction accuracy can be obtained from the difference between the predicted position of the person area and the position of the actual person area by predicting the position of the person area using a prediction method over a plurality of frames. FIG. 7 is a diagram illustrating an example of calculating the prediction accuracy. For example, the movement position of the person area predicted at time t-2 is (x ₂ (t-2), y ₂ (t-2)), and the position of the actual person area is (x ₁ (t-2)). , Y ₁ (t-2)). The movement position of the person area predicted at time t−1 is (x ₂ (t−1), y ₂ (t−1)), and the actual position of the person area is (x ₁ (t−1), y ₁ (t-1)). It is assumed that the movement position of the person area predicted at time t is (x ₂ (t), y ₂ (t)), and the actual position of the person area is (x ₁ (t), y ₁ (t)).

  The prediction accuracy at time t-2 is obtained from the following equations (8) and (9). The prediction accuracy at time t-1 is obtained from the following equations (10) and (11). The prediction accuracy at time t is obtained from the following equations (12) and (13).

_{x p1 = 1- {| x 1} (t-2) -x 2 (t-2) | / x 1 (t-2)} (8)
y _p1 = 1− {| y ₁ (t−2) −y ₂ (t−2) | / y ₁ (t−2)} (9)
_{x p2 = 1- {| x 1} (t-1) -x 2 (t-1) | / x 1 (t-1)} (10)
y _p2 = 1− {| y ₁ (t−1) −y ₂ (t−1) | / y ₁ (t−1)} (11)
x _p3 = 1− {| x ₁ (t) −x ₂ (t) | / x ₁ (t)} (12)
y _p3 = 1− {| y ₁ (t) −y ₂ (t) | / y ₁ (t)} (13)

Here, “| x ₁ (t−2) −x ₂ (t−2) |” represents the absolute value of x ₁ (t−2) −x ₂ (t−2).

For example, it is _assumed that the prediction accuracy of the human region is (x _p1 , y _p1 ), (x _p2 , y _p2 )... (X _pN , y _pN ) for the captured N frames of images. In this case, the prediction accuracy (p _x1 , p _y1 ) of the human region in N frames of images can be obtained from the following equations (14) and (15).

p _x1 = (x _p1 + x _p2 +... + x _pN ) / N (14)
p _y1 = (y _p1 + y _p2 +... + y _pN ) / N (15)

Further, it is assumed that the prediction accuracy of the M human regions is (p _x1 , p _y1 ), (p _x2 , p _y2 )... (P _xM , p _yM ). In this case, the prediction accuracy (p _x , p _y ) in M person regions can be obtained from the following equations (16) and (17).

p _x = (p _x1 + p _x2 +... + p _xM ) / M (16)
_{p y = (p y1 + p} y2 + ··· + p yM) / M (17)

In the imaging control apparatus 11 according to the present embodiment, the position of the person area is predicted in advance by the prediction method shown in the equations (6) and (7), and the movement position of the predicted person area is set to the actual person area. Compared with the position, the prediction accuracy (p _x , p _y ) is determined. Note that the prediction accuracy (p _x , p _y ) may be dynamically updated based on an error between the predicted movement position of the person area and the actual position of the person area in the captured image.

The prediction error can be calculated from the prediction accuracy. FIG. 8 is a diagram illustrating an example of calculating a prediction error. FIG. 8 shows the coordinates of the predicted movement position. For example, the movement position of the person area is predicted in the order of (5, 10), (10, 20), (15, 30) using the above-described equations (6) and (7). Since the movement position obtained from the above equations (6) and (7) is a predicted position, there is a case where a deviation (prediction error ε) occurs unlike the actual position of the person region. For example, a p _x prediction accuracy of the position of the person area is in the x-axis, a p _y in the y-axis, if the coordinates of the mobile position is (15, 30), in the x axis direction and y axis direction, the following In some cases, the deviations shown in equations (18) and (19) occur.

Deviation in the x-axis direction = 15 × (1−p _x ) (18)
Deviation in the y-axis direction = 30 × (1- _py ) (19)

  In this case, the prediction error ε is calculated from the following equation (20), for example.

  The imaging control unit 42 predicts movement positions at the next imaging timing for each of a plurality of selectable frame rates for a plurality of human regions of the captured image. For example, the shooting control unit 42 predicts the moving position at the next shooting timing for each of the plurality of frame rates using the above-described equations (6) and (7) for each person region. For example, when the selectable frame rates are 30 fps, 15 fps, and 5 fps, the imaging control unit 42 changes the frame rates f in Expressions (6) and (7) to “30”, “15”, and “5”, respectively. Thus, the movement position at the shooting timing of each frame rate is predicted. The imaging control unit 42 calculates the prediction error ε of the predicted movement position using the equations (18) to (20).

  The imaging control unit 42 calculates the distance between the predicted movement positions of each person area for each frame rate. For example, assume that the predicted movement position of the person area A is (x1, y1), and the prediction error is ε1. The predicted movement position of the person area B is (x2, y2), and the prediction error is ε2. In this case, the distance d between the movement positions of the person area A and the person area B is calculated from the following equation (21).

  The imaging control unit 42 determines whether the distance between the predicted movement positions of each person area is larger than the prediction error for each frame rate. For example, the imaging control unit 42 determines whether the distance d between the movement positions of the person area A and the person area B satisfies the following expression (22).

  d> ε1 + ε2 (22)

  When it is determined that the distance between the predicted movement positions is larger than the prediction error, the imaging control unit 42 determines the frame rate determined to be larger than the prediction error as the frame rate of the monitoring camera 12. . In addition, when there are a plurality of frame rates in which the distance between the predicted moving positions is larger than the prediction error, the imaging control unit 42 has the lowest frame among the frame rates in which the distance between the moving positions is larger than the prediction error. The rate is determined as the frame rate of the monitoring camera 12. In addition, when there is no frame rate at which the distance between the predicted movement positions is larger than the prediction error, the imaging control unit 42 selects the highest frame rate from among a plurality of selectable frame rates. And decide. The imaging control unit 42 transmits instruction information indicating the determined frame rate to the monitoring camera 12 to control the frame rate of the monitoring camera 12.

  FIG. 9 is a diagram illustrating an example of the distance between the movement positions of the person area. FIG. 9 shows a person area A and a person area B. In the person area A, the position at time t-1 is (x1 (t-1), y1 (t-1)). In the person area B, the position at time t−1 is (x2 (t−1), y2 (t−1)). In the example of FIG. 9, the distance between the movement positions predicted when the frame rate is f1 is d1, and the distance between the movement positions predicted when the frame rate is f2 is d2. The frame rate is assumed to be f1> f2. In the example of FIG. 9, the distance d1> ε1 + ε2 and the distance d2 <ε1 + ε2. In this case, the imaging control unit 42 controls the frame rate of the monitoring camera 12 to f1.

  By the way, for example, a portion close to the edge of the image is a region where the frequency of appearance of a person first is high. If the already detected human area in the image is located near the edge of the image, a part of the human area may be interrupted by the boundary of the image, and it is likely to be erroneously detected as a newly appearing human area.

  Therefore, when the movement position predicted at the determined frame rate is located in a region within a predetermined range from the edge of the image, the imaging control unit 42 selects the highest frame rate from among a plurality of selectable frame rates. Determine the frame rate. The imaging control unit 42 transmits instruction information indicating the determined frame rate to the monitoring camera 12 to control the frame rate of the monitoring camera 12.

  FIG. 10 is a diagram illustrating an example of a region within a predetermined range from the edge of the image. FIG. 10 shows the coordinates of the region X within a predetermined range from the edge of the image. In the example of FIG. 10, the region X is a region where the y-axis coordinate is less than or equal to ye, or greater than ys, and the x-axis coordinate is less than or equal to xs or greater than xe. For example, the coordinates of the movement position predicted at the determined frame rate are (xp, yp). When xp <xs or xe <xp, or when yp> ys or ye> yp, the imaging control unit 42 sets the highest frame rate among the selectable frame rates as the frame rate of the monitoring camera 12. decide. For example, if the selectable frame rate is 30 fps, 15 fps, and 5 fps, and the frame rate is determined to be 5 fps, the imaging control unit 42 determines that the movement position predicted at 5 fps is within the region X, 30 fps. And decide.

The area X may be fixed and preset. Further, the region X may be set by performing a test before actually operating the imaging control device 11. The position coordinates of the person newly detected in the lower part of the image by the specifying unit 43 to be described later are (x _u1 , y _u1 ), (x _u2 , _{yu 2} ), (x _u3 , _{yu 3} )... (X _uN , y _uN ). In this case, the coordinate ye defining the region X is calculated from the following equation (23).

ye = (x _u1 + x _u2 + x _u3 +... + x _uN ) / N (23)

  Note that N is the number of persons who appear from the lower part of the image.

Also, the position coordinates of the newly detected person in the image top by a particular unit _{43 (x up1, y up1)} , (x up2, y up2), (x up3, y up3) ··· (x upM, y _upM ). In this case, the coordinate ys defining the region X is calculated from the following equation (24).

_{ys = (y up1 + y up2} + y up3 + ··· + y upM) / M (24)

  Note that M is the number of persons appearing from the top of the image.

Further, the position coordinates of the person newly detected at the left part of the image by the specifying unit 43 are (x _l1 , y _l1 ), (x _l2 , y _l2 ), (x _l3 , y _l3 )... (X _lL , _ylL ). In this case, the coordinate xs defining the region X is calculated from the following equation (25).

xs = ( _xl1 + _xl2 + _xl3 + ... + _xlL ) / L (25)

  Note that L is the number of persons who appear from the left part of the image.

Further, the position coordinates of the person newly detected in the right part of the image by the specifying unit 43 are (x _r1 , y _r1 ), (x _r2 , y _r2 ), (x _r3 , y _r3 )... (X _rk , y _rK ). In this case, the coordinate xe defining the region X is calculated from the following equation (26).

xe = (x _r1 + x _r2 + x _r3 +... + x _rL ) / K (26)

  Note that K is the number of persons who appear from the right part of the image.

  Returning to FIG. 2, the identification unit 43 performs various types of identification. For example, the specifying unit 43 specifies a person area from the image of the image data acquired by the acquiring unit 40. The image of the image data acquired by the acquisition unit 40 is the newest image captured by the monitoring camera 12. FIG. 11 is a diagram illustrating an example of a flow for specifying a person area. The specifying unit 43 obtains a difference between the image acquired by the acquiring unit 40 and the image one frame before the image. For example, the specifying unit 43 obtains a background difference between the image at time t acquired by the acquisition unit 40 and the image at time t−1 stored in the storage unit 21. The specifying unit 43 specifies a candidate for the human region from the background difference image. For example, the specifying unit 43 specifies an area having a certain change from the image one frame before in the newest image as a candidate for the person area. The specifying unit 43 determines whether the specified person area candidate is a person. For example, the specifying unit 43 determines whether the candidate for the person area is a person using a discriminator that has learned the characteristics of the person area using the correct answer data of the person. For example, HOG (Histogram of Oriented Gradients) may be used as a feature amount used for discrimination of a person region, but other feature amounts may be used. The specifying unit 43 specifies the x-coordinate, y-coordinate, height, and width of the person area in the image from the person area determined to be a person.

  FIG. 12 is a diagram illustrating an example of a flow of determining a person area based on the overlapping rate. The specifying unit 43 calculates the overlapping rate between the person area that has been specified in the image one frame before and the person area that has been determined to be a person. For example, the specifying unit 43 calculates the overlapping rate with the person region of the image at time t−1 stored in the storage unit 21. When the duplication rate is equal to or higher than the predetermined threshold th, the specifying unit 43 associates the person area determined to be a person with the specified person area. For example, the specifying unit 43 registers the x coordinate, y coordinate, height, width, and shooting frame rate of the person area in the person tracking information 31 with the same person ID as that of the specified person area. Note that the threshold th may be a fixed value or may be adjustable from the outside by a person using the imaging control device 11. The frame rate may be received together with the image data from the monitoring camera 12 and stored, or the latest frame rate instructed to the monitoring camera 12 may be stored.

  On the other hand, when the duplication rate is less than the predetermined threshold th, the specifying unit 43 assigns a new person ID to the person area determined to be a person, and the assigned person ID uses the x coordinate, y coordinate, The height, width, and shooting frame rate are registered in the person tracking information 31. Note that the method for specifying the person region from the image is not limited to this, and any method may be used.

  The tracking unit 44 performs various types of tracking. For example, the tracking unit 44 calculates the movement speed of the person region from a plurality of captured images. For each person ID, the tracking unit 44 calculates the moving speed of the person area of the person ID from two or more past images stored in the storage unit 21. For example, for each person ID, the tracking unit 44 predicts the movement position of the person area in the latest photographed image based on the position of the person area in any image and the movement speed of the person area. FIG. 13 is a diagram illustrating an example of a flow for tracking a person area. The tracking unit 44 predicts the movement position of the person area in the image at time t based on the position of the person area in the image at time t−1 and the movement speed of the person area. The tracking unit 44 searches for a person region having a similar feature within a predetermined range based on the predicted movement position. In the example of FIG. 13, the tracking unit 44 determines r pixel matching ranges on the x-axis and the y-axis from the predicted movement position. The tracking unit 44 performs pattern matching within the matching range using the person area in the image at time t−1 as a template, and obtains a person area having similar characteristics. The tracking unit 44 determines a target area having the same size as the person area in the image at time t-1 within the matching range. The tracking unit 44 calculates the similarity by using the person region in the image at time t−1 as the template T and the target region in the matching range as the comparison target I. The tracking unit 44 repeats obtaining the similarity with the template T by moving the target region so as to cover the target region within the matching range. The tracking unit 44 determines whether or not the calculated maximum value of the similarity is equal to or greater than a predetermined threshold. When the maximum value is equal to or greater than a predetermined threshold, the tracking unit 44 associates the target area for which the similarity that has reached the maximum value is calculated with the person area of the template T. For example, the tracking unit 44 obtains the x coordinate, y coordinate, height, width, and shooting frame rate of the target region. The tracking unit 44 registers the x-axis, y-coordinate, height, width, and shooting frame rate of the target area in the person tracking information 31 with the same person ID as the person area of the template T, with the target area as the person area. The person area tracking method is not limited to this, and any method may be used. For example, the tracking unit 44 may define a matching range from the reference position in the image at time t with the position of the person region in the image at time t−1 as a reference, and pattern matching within the matching range. For example, the tracking unit 44 may determine a range within a predetermined distance from the reference position as the matching range in the image at time t. The predetermined distance may be changed according to the frame rate and the moving speed of the person area as a template. For example, the predetermined distance may be increased as the frame rate is lower and the moving speed of the person area serving as the template is faster, and may be decreased as the frame rate is higher and the moving speed of the person area serving as the template is slower.

[Process flow]
A flow of control processing in which the imaging control device 11 according to the first embodiment controls imaging of the monitoring camera 12 will be described. FIG. 14 is a flowchart illustrating an example of a control processing procedure according to the first embodiment. This extraction process is executed at a predetermined timing, for example, a timing at which execution is instructed by the administrator or a timing at which reception of image data from the monitoring camera 12 is started.

  As illustrated in FIG. 14, the acquisition unit 40 determines whether image data has been received from the monitoring camera 12 via the communication I / F unit 20 (S10). If no image data has been received (No at S10), the process proceeds to S10 again.

  On the other hand, when image data is received (Yes at S10), the calculation unit 41 executes a similarity calculation process (S11).

  FIG. 15 is a flowchart illustrating an example of a procedure of similarity calculation processing. This similarity calculation process is executed at a predetermined timing, for example, S11 of the control process.

  As illustrated in FIG. 15, the calculation unit 41 determines whether or not two or more person regions are included in an image with the latest time of the person tracking information 31 (S30). If the image does not include two or more person regions (No at S30), the similarity calculation process is terminated, and the process proceeds to S12 of the control process shown in FIG.

  On the other hand, when the image includes two or more person areas (Yes in S30), the calculation unit 41 determines a combination that combines two or more person areas two by two (S31). The calculation unit 41 selects one unprocessed combination from the combinations determined in S31 (S32).

  The calculation unit 41 calculates the similarity between two person regions that are combined (S33). The calculation unit 41 determines whether selection of all combinations determined in S31 is completed (S34). If selection of all combinations has not been completed (No at S34), the process proceeds to S32 described above. On the other hand, when the selection of all combinations is completed (Yes in S34), the similarity calculation process is terminated, and the process proceeds to S12 of the control process shown in FIG.

  Returning to FIG. 14, the calculation unit 41 executes a speed calculation process (S12).

  FIG. 16 is a flowchart illustrating an example of a procedure of speed calculation processing. This speed calculation process is executed at a predetermined timing, for example, S12 of the control process.

  As illustrated in FIG. 16, the calculation unit 41 determines whether or not two or more person regions are included in the image with the latest time of the person tracking information 31 (S50). If the image does not include two or more person areas (No at S50), the speed calculation process is terminated, and the process proceeds to S13 of the control process shown in FIG.

  On the other hand, when the image includes two or more person areas (Yes in S50), the calculation unit 41 determines whether there are person areas similar to each other in the person areas based on the similarity between the person areas (S51). If there is no person area similar to each other in the person area (No in S51), the speed calculation process is terminated, and the process proceeds to S13 of the control process shown in FIG.

  On the other hand, when there are person areas similar to each other in the person area (Yes in S51), the calculation unit 41 refers to the person tracking information 31 and specifies the movement amount of the person area for each person area (S52). For each person area, the calculation unit 41 calculates the movement speed from the movement amount of the person area (S53), ends the speed calculation process, and proceeds to S13 of the control process shown in FIG.

  Returning to FIG. 14, the imaging control unit 42 executes a frame rate control process (S13).

  FIG. 17 is a flowchart illustrating an example of the procedure of the frame rate control process. This frame rate control process is executed at a predetermined timing, for example, S13 of the control process.

  As illustrated in FIG. 17, the imaging control unit 42 determines whether or not two or more person regions are included in the image with the latest time of the person tracking information 31 (S70). When two or more person areas are not included in the image (No in S70), the process proceeds to S74 described later.

  On the other hand, when the image includes two or more person areas (Yes in S70), the imaging control unit 42 determines whether there are person areas similar to each other in the plurality of person areas based on the similarity between the person areas ( S71). If there is no person area similar to each other in the person area (No in S71), the process proceeds to S74 described later.

  On the other hand, when there are person areas similar to each other in the person area (Yes in S71), the shooting control unit 42 at the next shooting timing of each of a plurality of selectable frame rates for a plurality of person areas of the shot image. Each movement position is predicted (S72). The imaging control unit 42 determines the frame rate at which the distance between the predicted movement positions is larger than the prediction error as the frame rate of the monitoring camera 12 (S73).

  On the other hand, the imaging control unit 42 determines the lowest frame rate among the selectable frame rates as the frame rate of the monitoring camera 12 (S74).

  The imaging control unit 42 transmits instruction information indicating the determined frame rate to the monitoring camera 12 (S75), ends the frame rate control process, and proceeds to S14 of the control process illustrated in FIG.

  Returning to FIG. 14, the specifying unit 43 executes a person detection process (S <b> 14).

  FIG. 18 is a flowchart illustrating an example of a procedure of person detection processing. This person detection process is executed at a predetermined timing, for example, S14 of the control process.

  As illustrated in FIG. 18, the specifying unit 43 obtains a background difference between the image at time t acquired by the acquisition unit 40 and the image at time t−1 stored in the storage unit 21 (S100). The specifying unit 43 specifies an area having a certain change from the image one frame before in the newest image as a person area candidate (S101).

  The identification unit 43 determines whether the candidate for the person area is a person using a discriminator that has learned the characteristics of the person area using the correct answer data of the person (S102). The specifying unit 43 calculates the overlapping rate between the person area already specified in the image one frame before and the person area determined to be a person (S103). The identifying unit 43 determines whether or not the duplication rate is equal to or greater than the threshold th (S104). When the duplication rate is equal to or greater than the threshold th (Yes in S104), the specifying unit 43 registers information related to the person area with the same person ID as the specified person area in the person tracking information 31 (S105), and ends the person detection process. Then, the process proceeds to S15 of the control process shown in FIG. For example, the specifying unit 43 registers the x coordinate, y coordinate, height, width, and shooting frame rate of the person area in the person tracking information 31 as information about the person area.

  On the other hand, when the duplication rate is less than the predetermined threshold th (No in S104), the specifying unit 43 assigns a new person ID. The specifying unit 43 registers information related to the person area of the person area with the assigned person ID in the person tracking information 31 (S106), ends the person detection process, and proceeds to S15 of the control process illustrated in FIG.

  Returning to FIG. 14, the tracking unit 44 executes a person tracking process (S15).

  FIG. 19 is a flowchart illustrating an example of the procedure of the person tracking process. This person tracking process is executed at a predetermined timing, for example, S15 of the control process.

  As shown in FIG. 19, the tracking unit 44 moves the person area in the latest captured image based on the position of the person area in any image and the movement speed of the person area for each person ID. Is predicted (S120). The tracking unit 44 determines r pixel matching ranges on the x-axis and y-axis from the predicted movement position (S121). The tracking unit 44 performs pattern matching within the matching range using the person area in the image at time t−1 as a template, and obtains a person area having similar characteristics (S122). For example, the tracking unit 44 repeats obtaining the similarity with the template T by moving the target region so as to cover the target region within the matching range.

  When the calculated maximum value of the similarity is equal to or greater than a predetermined threshold, the tracking unit 44 associates the target area for which the similarity with the maximum value is calculated with the person area of the template T (S123). The tracking unit 44 sets the target area as a person area, registers information about the person area in the person tracking information 31 with the same person ID as the person area of the template T (S124), ends the person tracking process, and returns to FIG. The process proceeds to S16 in the control process shown.

  Returning to FIG. 14, the acquisition unit 40 determines whether or not to end the process (S <b> 16). For example, when an instruction to end the process is received, the acquisition unit 40 determines that the process is to end. If the process is finished (Yes at S16), the control process is finished. On the other hand, if the process is not finished (No at S16), the process proceeds to S10 described above.

[effect]
The imaging control device 11 specifies a person area from the image captured by the monitoring camera 12. When a plurality of person areas are specified in the image captured by the monitoring camera 12, the imaging control device 11 determines the positional relationship and the moving speed of each of the plurality of person areas from the plurality of images captured by the monitoring camera 12. calculate. The imaging control device 11 controls the frame rate of the monitoring camera 12 based on the positional relationship in the identified images of the plurality of person areas and the moving speed of the plurality of person areas. In this way, the shooting control apparatus 11 can predict where the person area is located at the next shooting by using the positions and moving speeds of the plurality of person areas. Thereby, the imaging control device 11 can predict whether or not the position of the person area at the next imaging is likely to cause a decrease in tracking accuracy. 20 and 21 are diagrams illustrating an example of the movement of the person area. In FIG. 20, the person area A and the person area B are moved closer to each other, and it is predicted that the person area A and the person area B overlap each other at the next shooting. If the person area A and the person area B overlap in the image, erroneous correspondence is likely to occur in tracking the person. Therefore, FIG. 20 is a situation in which a decrease in tracking accuracy is likely to occur. In FIG. 21, the person area A and the person area B are moved in parallel, and the distance between the person area A and the person area B is maintained even during the next shooting. Therefore, FIG. 20 is a situation in which a decrease in tracking accuracy hardly occurs. 20 and 21, the positions of the person area A and the person area B are the same. However, the imaging control device 11 can determine whether the tracking accuracy is likely to decrease during the next imaging by using the position and moving speed of the person area. When the next shooting is in a situation where the tracking accuracy is likely to be lowered, the imaging control device 11 can suppress the occurrence of erroneous correspondence or erroneous detection by changing the frame rate of the monitoring camera 12 to be high, for example. A decrease in accuracy can be suppressed. In addition, when the next shooting is a situation in which erroneous detection is unlikely to occur, the shooting control device 11 can control the processing load by, for example, keeping the frame rate of the monitoring camera 12 low.

  In addition, when a plurality of person areas are specified in the captured image, the imaging control device 11 predicts the movement positions of the plurality of person areas at the next imaging timing for each of the predetermined plurality of frame rates. The imaging control device 11 predicts the distance between the predicted movement positions from the difference between the position of the person area predicted from the positional relationship and movement speed of each of the plurality of person areas and the position of the actual person area. The frame rate of the monitoring camera 12 is controlled to a frame rate larger than the error. A situation in which the distance between the movement positions of the person area is equal to or less than the prediction error is a situation in which erroneous correspondence or erroneous detection is likely to occur and tracking accuracy is likely to be reduced. Therefore, the imaging control device 11 predicts the position of each person area at the next imaging for each of a plurality of frame rates. The imaging control device 11 changes the frame rate so that the distance between the individual areas is larger than the prediction error. FIG. 22 is a diagram illustrating an example of changing the frame rate. The left side of FIG. 22 shows an image captured by the monitoring camera 12 at time t at the lowest frame rate among a plurality of frame rates. The images from time t-1 to time t include a person area A and a person area B. On the left side of FIG. 22, it is predicted that the person area A and the person area B overlap at the time t + 1, which is the next shooting timing. In this case, the imaging control device 11 changes the frame rate of the monitoring camera 12 to a frame rate in which the distance between the person area A and the person area B is larger than the prediction error. The right side of FIG. 22 shows an image shot at time t ′ when the shooting control device 11 changes the frame rate of the monitoring camera 12 at time t. In the image at the time t ′, the distance between the person area A and the person area B is kept more than the prediction error ε. Thereby, since the imaging | photography control apparatus 11 can suppress generation | occurrence | production of a miscorrespondence and a misdetection, it can suppress the fall of tracking accuracy. On the right side of FIG. 22, the distance between the person area A and the person area B at the time t ′ + 1 predicted at the lowest frame rate at the time t ′ is larger than the prediction error ε. Therefore, the imaging control device 11 changes the frame rate of the monitoring camera 12 to the lowest frame rate at time t ′. Thereby, the imaging control apparatus 11 can control the processing load.

  In addition, when there are a plurality of frame rates in which the distance between the moving positions is larger than the prediction error, the imaging control device 11 has the lowest frame rate among the plurality of frame rates in which the distance between the moving positions is larger than the prediction error. The frame rate of the monitoring camera 12 is controlled. Thereby, the imaging control device 11 can control the processing load while suppressing a decrease in tracking accuracy.

  In addition, when the movement position predicted at the frame rate in which the distance between the movement positions is larger than the prediction error is located in a predetermined range from the edge of the image, the imaging control device 11 performs the following process. The imaging control device 11 controls the frame rate of the monitoring camera 12 to the highest frame rate among the plurality of frame rates. A portion close to the edge of the image is likely to be erroneously detected when a part of the person region is interrupted or a newly appearing person region. On the other hand, when the frame rate is increased, the surveillance camera 12 captures images with a short imaging cycle, so that changes between captured images are reduced. When the change between images is small, the tracking accuracy is improved. The imaging control device 11 can suppress a decrease in tracking accuracy by increasing the frame rate in a situation where the movement position of the person region is close to the edge and erroneous detection is likely to occur.

  In addition, when a plurality of person regions are not specified in the captured image, the imaging control device 11 controls the frame rate of the monitoring camera 12 to the lowest frame rate among a plurality of predetermined frame rates. A situation in which a plurality of person regions are not included in a photographed image is a situation in which it is difficult for erroneous correspondence or erroneous detection to occur, and a decrease in tracking accuracy is difficult to occur. The imaging control device 11 controls the frame rate of the monitoring camera 12 to the lowest frame rate when it is difficult for the tracking accuracy to decrease. Thereby, the imaging control apparatus 11 can control the processing load.

  In addition, the imaging control device 11 controls the frame rate of the monitoring camera 12 when a plurality of person areas are specified in the captured image and the person areas are similar to each other. If there are person regions that are similar to each other in the captured image, erroneous correspondence is likely to occur. The imaging control device 11 can avoid a situation in which the tracking accuracy is likely to be lowered by changing the frame rate of the monitoring camera 12 when the next imaging is in a situation where the tracking accuracy is likely to be reduced. Can be suppressed.

  Further, the imaging control device 11 predicts the movement position of the person area in the captured image based on the position of the person area and the moving speed of the person area. The imaging control device 11 tracks a person by associating person regions with similar characteristics within a predetermined range based on the predicted movement position. Thereby, since the imaging | photography control apparatus 11 can associate the person area | region of the same person accurately, the tracking precision of a person improves.

  Further, the imaging control device 11 specifies a person area from the difference between the captured image and the image one frame before the image. The surveillance camera 12 is installed in various situations such as outdoors. The person in the image has a variety of situations such as body shape, clothing color, obstacles, brightness change, and the like, and it is difficult to specify the person area. However, the imaging control device 11 specifies the person area from the difference between the captured image and the image one frame before the image, so that the person area is independent of the installation status of the monitoring camera 12 and the situation of the person. Can be accurately identified.

  In addition, when the overlapping rate between the person area identified from the difference and the person area one frame before is equal to or greater than a predetermined threshold, the imaging control device 11 determines the person area identified from the difference as the person area of the image one frame before And identify the person area of the same person. When the overlapping rate is less than the predetermined threshold, the imaging control device 11 specifies the person area specified from the difference as the person area of the new person. Thereby, the imaging control device 11 can accurately identify the person area of the specified person or the person area of a new person from the captured image.

  Although the embodiments related to the disclosed apparatus have been described so far, the disclosed technology may be implemented in various different forms other than the above-described embodiments. Therefore, another embodiment included in the present invention will be described below.

  For example, the order of the processes in the above embodiment is an example, and the order of some processes may be changed. For example, in the control process, the order of similarity calculation process, speed calculation process, frame rate control process, person detection process, and person tracking process may be switched.

  FIG. 23 is a flowchart illustrating an example of a procedure of control processing according to the second embodiment. Since the control processing according to the second embodiment is partially the same as the control processing shown in FIG. 14, the same portions are denoted by the same reference numerals, and different portions are mainly described. In the control process according to the second embodiment, the person detection process and the person tracking process are performed before the similarity calculation process, the speed calculation process, and the frame rate control process.

  As shown in FIG. 23, when image data is received (Yes at S10), the imaging control unit 42 performs a person detection process on the image at time t acquired by the acquisition unit 40 (S11A). Next, the imaging control unit 42 executes a person tracking process (S12A). As a result, a person region is detected from the image at time t acquired by the acquisition unit 40 and registered in the person tracking information 31.

  The calculation unit 41 performs similarity calculation processing on the image at time t acquired by the acquisition unit 40 (S13A). The calculation unit 41 performs a speed calculation process on the image at time t (S14A). The imaging control unit 42 performs a frame rate control process using a plurality of images including the image at time t (S15A). As shown in FIG. 23, the control process can control photographing even when the order of some processes is changed.

  In the above embodiment, the imaging control unit 42 determines the frame rate of the monitoring camera 12 when a plurality of person areas are specified in the captured image and the person areas are similar to each other. The case of controlling was illustrated. However, it is not limited to these. For example, the imaging control unit 42 may control the frame rate of the monitoring camera 12 when a plurality of person regions are specified in the captured image. That is, the imaging control unit 42 may control the frame rate of the monitoring camera 12 on the condition that a plurality of person areas are specified even if there is no similar person area.

  Further, the shooting control unit 42 may control the frame rate of the monitoring camera 12 to be high when a plurality of person areas are specified in the shot image and there are person areas similar to the plurality of person areas. . That is, the imaging control unit 42 may control the frame rate of the monitoring camera 12 on the condition that there are similar person areas regardless of the distance between the person areas. For example, the imaging control unit 42 may control the frame rate higher as the similarity is higher. The imaging control unit 42 determines the similarity threshold in stages. For example, the imaging control unit 42 determines thresholds t1 and t2 for similarity. The threshold values t1 and t2 are assumed to be t1> t2. The imaging control unit 42 determines the frame rate as 30 fps when the similarity is greater than or equal to the threshold t1, determines the frame rate as 15 fps when the similarity is less than the threshold t1 and greater than or equal to the threshold t2, and when less than the threshold t2. The frame rate may be determined to be 5 fps.

  FIG. 24 is a flowchart illustrating an exemplary procedure of a frame rate control process according to the second embodiment. The frame rate control process according to the second embodiment is partially the same as the frame rate control process illustrated in FIG. 17, and thus the same parts are denoted by the same reference numerals, and different parts are mainly described.

  As shown in FIG. 24, when there are person areas similar to each other in the person area (Yes in S71), the imaging control unit 42 determines the frame rate according to the similarity between the person areas (S73A). For example, the imaging control unit 42 determines the frame rate to be 30 fps when the maximum similarity between person areas is equal to or greater than the threshold value t1. The imaging control unit 42 determines the frame rate to be 15 fps when the maximum value is less than the threshold value t1 and greater than or equal to the threshold value t2. The imaging control unit 42 determines the frame rate to be 5 fps when it is less than the threshold value t2. By the way, although the maximum similarity between person regions is not equal to or greater than the threshold t1, there are a plurality of similarities between person regions that are less than the threshold t1 and greater than or equal to the threshold t2, and the tracking accuracy as a whole is likely to decrease. is there. Therefore, the imaging control unit 42 may add a predetermined number of similarities from the top in descending order of similarity between person regions, and determine the frame rate according to the added value of similarities. Thereby, although the maximum value of the similarity is not large, even when the tracking accuracy tends to be reduced as a whole, it is possible to suppress the deterioration of the tracking accuracy.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific state of distribution / integration of each device is not limited to the one shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. For example, the acquisition unit 40, the calculation unit 41, the imaging control unit 42, the specifying unit 43, and the tracking unit 44 may be appropriately integrated. Further, the processing of each processing unit may be appropriately separated into a plurality of processing units. Further, all or any part of each processing function performed in each processing unit can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware by wired logic. .

[Shooting control program]
The various processes described in the above embodiments can also be realized by executing a program prepared in advance on a computer system such as a personal computer or a workstation. Therefore, in the following, an example of a computer system that executes a program having the same function as in the above embodiment will be described. FIG. 25 is a diagram illustrating a computer that executes an imaging control program.

  As shown in FIG. 25, the computer 300 includes a central processing unit (CPU) 310, a hard disk drive (HDD) 320, and a random access memory (RAM) 340. These units 310 to 340 are connected via a bus 400.

  The HDD 320 stores in advance a shooting control program 320 a that exhibits the same functions as the processing units of the acquisition unit 40, the calculation unit 41, the shooting control unit 42, the specifying unit 43, and the tracking unit 44. Note that the photographing control program 320a may be separated as appropriate.

  The HDD 320 stores various types of information. For example, the HDD 320 stores various data used for the OS and processing.

  The CPU 310 reads out and executes the shooting control program 320a from the HDD 320, thereby executing operations similar to those of the acquisition unit 40, the calculation unit 41, the shooting control unit 42, the specifying unit 43, and the tracking unit 44. That is, the imaging control program 320a performs the same operations as the acquisition unit 40, the calculation unit 41, the imaging control unit 42, the specifying unit 43, and the tracking unit 44.

  Note that the above-described shooting control program 320a is not necessarily stored in the HDD 320 from the beginning.

  For example, the program is stored in a “portable physical medium” such as a flexible disk (FD), a CD-ROM, a DVD disk, a magneto-optical disk, or an IC card inserted into the computer 300. Then, the computer 300 may read and execute the program from these.

  Furthermore, the program is stored in “another computer (or server)” connected to the computer 300 via a public line, the Internet, a LAN, a WAN, or the like. Then, the computer 300 may read and execute the program from these.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Supplementary note 1) A person area is specified from an image taken by a photographing apparatus,
When a plurality of person areas are specified in the captured image, the positional relationship and the moving speed of each of the plurality of person areas are calculated from the plurality of images captured by the imaging device, respectively.
An imaging control program that causes a computer to execute a process for controlling a frame rate of the imaging apparatus based on a positional relationship of the plurality of identified human areas in the image and a moving speed of the multiple human areas.

(Supplementary Note 2) When a plurality of person areas are specified in the captured image, the control process predicts the movement positions of the plurality of person areas at the next shooting timing for each of a plurality of predetermined frame rates. The distance between the predicted movement positions is larger than the prediction error obtained from the difference between the position of the person area predicted from the positional relationship and movement speed of each of the plurality of person areas and the position of the actual person area. The imaging control program according to appendix 1, wherein the frame rate of the imaging apparatus is controlled to a large frame rate.

(Supplementary Note 3) When there are a plurality of frame rates in which the distance between the moving positions is larger than a prediction error, the processing to be controlled includes a plurality of frame rates in which the distance between the moving positions is larger than the prediction error. The shooting control program according to appendix 2, wherein the frame rate of the shooting apparatus is controlled to the lowest frame rate.

(Supplementary Note 4) When the movement position predicted at a frame rate in which the distance between the movement positions is larger than the prediction error is located in a region within a predetermined range from the edge of the image, the process to be controlled is the plurality of frame rates. The shooting control program according to appendix 2 or 3, wherein the frame rate of the shooting device is controlled to the highest frame rate.

(Supplementary Note 5) When the plurality of human regions are not specified in the captured image, the control process controls the frame rate of the imaging device to the lowest frame rate among a plurality of predetermined frame rates. The imaging control program according to any one of Supplementary notes 1 to 4, which is a feature.

(Additional remark 6) The said process to control controls the frame rate of the said imaging device, when a several person area | region is specified to the image | photographed image, and there exist a person area | region similar to the said several person area | region. The imaging control program according to any one of appendices 1 to 5, characterized in that:

(Supplementary note 7) Based on the position of the person area and the movement speed of the person area, the movement position of the person area in the captured image is predicted, and the feature is within a predetermined range based on the predicted movement position. The imaging control program according to any one of appendices 1 to 6, further causing a computer to execute a process of tracking a person by associating similar person regions.

(Supplementary note 8) Any one of Supplementary notes 1 to 7, wherein the specifying process specifies a person region from a difference between the photographed image and an image photographed immediately before the image. The shooting control program described in 1.

(Supplementary Note 9) When the overlapping rate between the person area specified from the difference and the person area of the image taken immediately before is equal to or greater than a predetermined threshold, the person area specified from the difference is set to the previous one A process of identifying a person area of the same person as the person area of the image taken in the image and identifying the person area identified from the difference as a person area of a new person when the overlap ratio is less than a predetermined threshold. The imaging control program according to appendix 8, which is executed by a computer.

(Appendix 10) A person region is identified from an image photographed by the photographing device,
When a plurality of person areas are specified in the captured image, the positional relationship and the moving speed of each of the plurality of person areas are calculated from the plurality of images captured by the imaging device, respectively.
A photographing control method, wherein the computer executes a process of controlling a frame rate of the photographing device based on a positional relationship in the image of the plurality of identified human regions and a moving speed of the plural human regions.

(Additional remark 11) The specific part which specifies a person area | region from the image image | photographed with the imaging device,
When the specifying unit specifies a plurality of person regions in the captured image, the calculation calculates the positional relationship and the moving speed of each of the plurality of person regions from the plurality of images captured by the photographing device. And
A control unit for controlling a frame rate of the photographing apparatus based on a positional relationship in the image of the plurality of person regions specified by the specifying unit and a moving speed of the plurality of person regions calculated by the calculation unit;
An imaging control apparatus comprising:

(Supplementary Note 12) A person region is specified from an image photographed by the photographing apparatus,
When a plurality of person areas are specified in the photographed image and there are person areas similar to the plurality of person areas, the computer is caused to execute a process for controlling the frame rate of the photographing apparatus to be high. Shooting control program.

10 System 11 Imaging Control Device 12 Monitoring Camera 21 Storage Unit 22 Control Unit 30 Image Data 31 Person Tracking Information 40 Acquisition Unit 41 Calculation Unit 42 Imaging Control Unit 43 Identification Unit 44 Tracking Unit

Claims (12)

The person area is identified from the image taken by the photographing device,
When a plurality of person areas are specified in the captured image, the positional relationship and the moving speed of each of the plurality of person areas are calculated from the plurality of images captured by the imaging device, respectively.
An imaging control program that causes a computer to execute a process for controlling a frame rate of the imaging apparatus based on a positional relationship of the plurality of identified human areas in the image and a moving speed of the multiple human areas. The control process predicts a movement position of the plurality of person areas at a next shooting timing at each of a plurality of predetermined frame rates when a plurality of person areas are specified in the captured image. The distance between the moving positions is larger than the prediction error obtained from the difference between the position of the person area predicted from the positional relationship and the moving speed of each of the plurality of person areas and the position of the actual person area. The shooting control program according to claim 1, wherein a frame rate of the shooting apparatus is controlled. In the case where there are a plurality of frame rates in which the distance between the moving positions is larger than the prediction error, the processing to be controlled is the lowest frame rate among the plurality of frame rates in which the distance between the moving positions is larger than the prediction error. The imaging control program according to claim 2, further comprising: controlling a frame rate of the imaging apparatus. The control process is performed when the movement position predicted at a frame rate in which the distance between the movement positions is larger than the prediction error is located in a region within a predetermined range from the edge of the image. The shooting control program according to claim 2, wherein the frame rate of the shooting apparatus is controlled to a high frame rate. The control process includes controlling the frame rate of the imaging device to a lowest frame rate among a plurality of predetermined frame rates when a plurality of person regions are not specified in the captured image. Item 5. The shooting control program according to any one of Items 1 to 4. The control process includes controlling a frame rate of the photographing apparatus when a plurality of person areas are specified in the photographed image and the person areas are similar to each other. The photography control program according to any one of claims 1 to 5. Based on the position of the person area and the movement speed of the person area, the movement position of the person area in the captured image is predicted, and the person area having similar characteristics within a predetermined range based on the predicted movement position The photographing control program according to claim 1, further causing the computer to execute processing for tracking a person in association with each other. The said process to specify specifies a person area | region from the difference of the said image | photographed image and the image image | photographed immediately before the said image. The Claim 1 characterized by the above-mentioned. Shooting control program. When the overlapping ratio between the person area specified from the difference and the person area of the image taken immediately before is equal to or greater than a predetermined threshold, the person area specified from the difference was taken before the one A person area of the same person as the person area of the image is specified, and if the overlap ratio is less than a predetermined threshold, the computer further executes a process of specifying the person area specified from the difference as a person area of a new person The imaging control program according to claim 8, wherein: The person area is identified from the image taken by the photographing device,
When a plurality of person areas are specified in the captured image, the positional relationship and the moving speed of each of the plurality of person areas are calculated from the plurality of images captured by the imaging device, respectively.
A photographing control method, wherein the computer executes a process of controlling a frame rate of the photographing device based on a positional relationship in the image of the plurality of identified human regions and a moving speed of the plural human regions. A specifying unit for specifying a person area from an image taken by a photographing device;
When the specifying unit specifies a plurality of person regions in the captured image, the calculation calculates the positional relationship and the moving speed of each of the plurality of person regions from the plurality of images captured by the photographing device. And
A control unit for controlling a frame rate of the photographing apparatus based on a positional relationship in the image of the plurality of person regions specified by the specifying unit and a moving speed of the plurality of person regions calculated by the calculation unit;
An imaging control apparatus comprising: The person area is identified from the image taken by the photographing device,
When a plurality of person areas are specified in the photographed image and there are person areas similar to the plurality of person areas, the computer is caused to execute a process for controlling the frame rate of the photographing apparatus to be high. Shooting control program.

Images