JP2019219804A - Object detection device and object detection method - Google Patents

Abstract

To provide an object detection device and an object detection method which can improve accuracy in detecting an object located deep in an imaging space with less increase in cost for object detection.SOLUTION: A human head detection device 6 comprises: a detection region setting unit 35 which sets a detection region for person detection to an input image; a preprocessing unit 32 which subjects the input mage to preprocessing including projective transformation; a preprocessing parameter calculation unit 37 which generates a parameter for use in the preprocessing; and a human head detection unit 34 which detects a person from the image which is subjected to the preprocessing by using a prescribed detection model 42. The preprocessing parameter calculation unit 37 generates the parameter for the preprocessing including the projective transformation on the basis of a size on the image of a human head which is calculated from a camera parameter 44 at the time of imaging with a camera on the assumption that the human head has a fixed size, and a recommended size on the image of the human head which can be detected by using the detection model.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an object detection device and an object detection method for detecting an object such as a person using a camera or the like.

  When an object such as a person is detected by a surveillance camera, it is required that the object can be correctly detected regardless of the size and orientation of the object in a captured image. As one of the techniques, Patent Document 1 discloses that after projective transformation is performed on a captured image, object detection is performed by means such as machine learning. That is, in order to determine the posture of an object such as a person, the characteristics of a predetermined object in a specific posture are learned using a learning image taken from a specific direction, and a plurality of possible postures of the predetermined object are assumed. For each assumed posture, the input image is subjected to projective transformation for converting an image of a predetermined object in the posture into an image of a specific posture. Then, a window area is set in the input image that has been subjected to the projective transformation, and a score that is a degree at which the characteristic of the predetermined object in the specific posture appears in the window area is calculated. It is configured to determine that an image has been captured.

JP 2017-049676 A

  In recent years, the resolution of surveillance cameras has been improved, and it has become possible to perform object detection with high accuracy using deep learning represented by a convolutional neural network (CNN). The CNN basically inputs a rectangular image and detects a predetermined object from the image. At that time, the calculation becomes complicated, and the learning and detection are performed with a small image in a deep learning model that is often used. In an SSD (Single Shot Multibox Detector) which is an object detection algorithm, the size of an object that can be detected depends on a model at the time of learning, and only an object having a certain size or more can be detected. That is, the size of the detectable object in the image depends on the model size in the image at the time of learning (hereinafter, referred to as a minimum detection size).

  In the case of a large image with a high resolution, it is possible to detect an object deep in the shooting space, but the processing load (hereinafter, detection cost) increases. Therefore, the object detection is performed after the image is reduced. However, the size of the object at the back approaches the minimum detection size, and the detection accuracy is reduced. On the other hand, since the size of the object in front of the shooting space is very large compared to the minimum detection size, unnecessary cost is consumed. As described above, in a deep imaging space, it has been difficult to achieve both reduction in the cost of detecting an object and improvement in the detection accuracy.

  The technique described in Patent Document 1 is effective for a change in the posture of an object, but does not particularly consider a change in the image size of the object. That is, even if the same object is used, the fact that the size in the image changes according to the distance from the camera in the shooting space is not considered. Further, in Patent Document 1, it is necessary to repeatedly perform projective transformation corresponding to each of a plurality of postures, and there is a problem that the cost increases in object detection using deep learning.

  An object of the present invention is to provide an object detection device and an object detection method that improve the detection accuracy of an object located deep in an imaging space while suppressing an increase in the cost of object detection.

  The object detection device of the present invention includes a detection area setting unit that sets a detection area for detecting an object in an input image, a preprocessing unit that performs preprocessing including projective transformation on the input image, and a preprocessing unit. The apparatus includes a preprocessing parameter calculation unit that generates a parameter, and an object detection unit that detects an object from an image that has been preprocessed using a predetermined detection model. Here, the pre-processing parameter calculation unit estimates the size of the object on the image of the object calculated from the camera parameters at the time of photographing the camera, assuming that the object has a certain size, and the size of the image of the object that can be detected using the detection model. Based on the recommended size, parameters for preprocessing including projective transformation are generated.

  Further, the object detection method of the present invention includes a step of setting a detection area for detecting the object in an input image, a step of performing preprocessing including projective transformation on the input image, and generating a parameter used in the preprocessing. And a step of detecting an object from an image pre-processed using a predetermined detection model. Here, in the pre-processing parameter calculation step, the size of the object on the image of the object is calculated from the camera parameters at the time of photographing the camera, assuming that the object has a fixed size, Generate parameters for preprocessing including projective transformation in comparison with the recommended size.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to improve the detection precision of the object in the back of imaging space, suppressing the increase in the cost of object detection.

The figure which shows the whole structure of a people counting system. FIG. 2 is a diagram illustrating a hardware configuration of a human head detection device. FIG. 1 is a diagram illustrating a system configuration of a human head detection device. The figure which shows the whole operation flow of a person head detection apparatus. The figure which shows the flow of advance setting (S1). FIG. 4 is a diagram illustrating an example of setting camera parameters. The figure which shows the data structure of a detection model attribute. The figure which shows the screen of a detection area setting. The figure which shows the data structure of detection area information. The figure which shows the flow of preprocessing parameter generation (S2). The figure which shows the data structure of a head size list on a screen. FIG. 6 is a diagram showing a data structure of a preprocessing parameter 46. The figure which shows the confirmation screen of a pre-processing parameter. The figure which shows the flow of pre-processing (S3). The figure which shows the flow of a number measurement process (S4). The figure which shows the example of a display of a person detection result. The figure which shows the data structure of a person detection result.

  Hereinafter, as an embodiment of the object detection device of the present invention, a person head detection device that detects a person's head and a number measurement system that uses the device to measure the number of people traveling on a road or the like will be described.

  FIG. 1 is a diagram showing the overall configuration of the number measurement system. The people counting system 1 is a system that detects a person 3 passing through a specific area 2 such as a road, and saves the results of the detected number of people in chronological order. The people counting system 1 monitors a surveillance camera 4 for photographing a specific area 2, a network 5 such as a LAN (Local Area Network), a WAN (Wide Area Network), and a VPN (Virtual Private Network), and monitoring via the network 5. A human head detecting device 6 that receives the image of the camera 4, detects the head of the person 3 in the image, and measures the number of detected persons. Further, it is provided with a video display device 7 for displaying an image of the monitoring camera 4 and a detection result by the human head detection device 6 in real time, and a head count storage device 8 for storing detected head count information for each time. You. Since the purpose of this system is to measure the number of people, the human head detecting device 6 increases the processing efficiency as a method of detecting the head of the person 3. Hereinafter, the details of the human head detection device 6 will be described.

  FIG. 2 is a diagram illustrating a hardware configuration of the human head detection device 6. The human head detecting device 6 includes a CPU (Central Processing Unit) 11, a storage unit 12, a memory 20, an input / output unit 21, a communication unit 22, and a bus 19 connecting these.

  The CPU 11 is a unit that executes various calculations. The CPU 11 executes various processes by executing a predetermined program loaded from the storage unit 12 to the memory 20. The memory 20 temporarily stores programs executed by the CPU 11 and data necessary for executing the programs.

The storage unit 12 is a nonvolatile storage device such as a hard disk (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory capable of storing digital information. The storage unit 12 stores the following programs and data.
The number-of-people measurement program 13 describes instructions for all computers related to the number-of-people measurement. The CPU 11 develops the number-of-people measurement program 13 in the memory 20 and performs various processes of the number-of-people measurement.

The detection model DB (database) 14 stores parameters for identifying an object. For example, when an object is detected using an SVM (Support Vector Machine) identifier, the detection model DB 14 stores a support vector for distinguishing the feature amount between the object and another object. When performing object detection using the CNN, the detection model DB 14 stores parameters indicating the structure of the CNN network and initialization values of the learned model of the CNN network. In the present embodiment, description will be made on the assumption that model information of the CNN network is stored.
The detection model attribute DB 15 stores attributes of the model stored in the detection model DB 14, such as an inputtable image size and a detectable head size.

The camera parameter DB 16 stores installation information such as the height and depression angle of the monitoring camera 4 and shooting parameters of the camera such as an angle of view and a focal length.
The detection area setting DB 17 stores information for setting an area where the person 3 is to be detected in the specific area 2.

The pre-processing parameter DB 18 stores parameters for performing pre-processing such as projective transformation on the input image from the monitoring camera 4 to improve detection accuracy.
The input / output unit 21 inputs a user operation and performs signal conversion of an image or data transmitted / received to / from an external device. For example, the input / output unit 21 includes a graphic board, a video card, and the like, and converts an image of the monitoring camera 4, a result of the measurement of the number of persons, and the like into a signal that can be displayed on the video display device 7.
The communication unit 22 receives an image of the monitoring camera 4 via the network 5 and transmits a person detection result to the video display device 7 and the number history storage device 8.

  FIG. 3 is a diagram illustrating a system configuration of the human head detection device 6. The human head detection device 6 includes an image input unit 31, a preprocessing unit 32, a feature amount extraction unit 33, a human head detection unit 34, a detection area setting unit 35, an on-screen head size calculation unit 36, a preprocessing parameter calculation It comprises a unit 37, a detection result integration unit 38, an operation input unit 39, an output control unit 40, and program operation data 41.

  First, the program operation data 41 will be described. The data is loaded from the storage unit 12 by the CPU 11 and includes the following data. The detection model 42 is data of the detection model DB 14, and the detection model attribute 43 is data of the detection model attribute DB 15. The camera parameter 44 is data of the camera parameter DB 16, and the detection area information 45 is data of the detection area setting DB 17. The pre-processing parameter 46 is data of the pre-processing parameter DB 18.

The image input unit 31 receives encoded image data from the surveillance camera 4 via the network 5, decodes the image data, and converts the image data into an image in frame units.
The pre-processing unit 32 uses the pre-processing parameter 46 to perform pre-processing such as image extraction, reduction, and projective transformation on the camera image (also referred to as an input image) of the image input unit 31. The pre-processing is performed to accurately detect a person in the back of the shooting space.

The feature amount extraction unit 33 extracts a feature amount from the image processed by the preprocessing unit 32 using a CNN network or the like.
The human head detection unit 34 determines the presence or absence of a human head using the detection model 42 and the feature amount extracted by the feature amount extraction unit 33, and if there is a head, furthermore, the center position of the head and the head Calculate the size (width and height).

  The detection area setting section 35 sets which area of the camera image from the image input section 31 should be detected. If the size of the camera image (horizontal x vertical) is large, the image is reduced to an appropriate image size for detection using CNN. At this time, if the detection area is not set, the whole camera image is reduced to a predetermined size, and as a result, the head size of the person is too small and the detection accuracy is reduced. Therefore, by setting the detection area, the reduction ratio of the image can be suppressed, and a decrease in detection accuracy can be avoided. Further, by setting the detection area, it is possible to know the area to be emphasized, and to calculate a preprocessing parameter specialized for that area.

  The on-screen head size calculation unit 36 assumes that a person of a predetermined size (height, head size, etc.) is standing on the ground at each position on the screen, and Calculate the size on the screen. In this calculation, the camera parameters 44 of the monitoring camera 4 are used. That is, the size of the person's head seen in the screen is determined from the geometrical relationship between the position of the camera and the person, and the smaller the distance from the camera, the smaller it looks.

The preprocessing parameter calculation unit 37 calculates a preprocessing parameter 46 used when the preprocessing unit 32 performs preprocessing such as projective transformation based on the calculation result of the on-screen head size calculation unit 36. The pre-processing parameter 46 also includes information on an image cut-out area and a reduction ratio.
Based on the detection result of the human head detection unit 34, the detection result integration unit 38 outputs information on the presence or absence of a person and the coordinates of the detected person (center coordinates and coordinates of a rectangular mark superimposed on the human head. , The coordinates in the image before the pre-processing) are calculated, and these are integrated to output a person detection result.

The operation input unit 39 is a device such as a keyboard and a mouse, and identifies a user operation and converts it into an operation command.
The output control unit 40 converts the person detection result from the detection result integration unit 38 and the camera image from the image input unit 31 into a format that can be visually recognized by the user, and causes the video display device 7 to display the converted image. Further, the person detection result is output to the number history storage device 8 and stored.

Next, the operation of the human head detection device 6 will be described.
FIG. 4 is a diagram illustrating an overall operation flow of the human head detection device 6. The operation flow is roughly divided into four stages, and is performed in the order of pre-setting (S1), generation of pre-processing parameters (S2), pre-processing (S3), and measurement of the number of people (S4). The outline is as follows.

In the preliminary setting (S1), conditions such as a detection model and camera parameters used for human detection are set.
In the pre-processing parameter generation (S2), the size of the human head on the screen is calculated from the camera parameters, and the pre-processing parameters including the projective transformation are generated based on the size.
In the preprocessing (S3), preprocessing including projective transformation is performed on the input image using the preprocessing parameters.
In the number of people measurement (S4), a human head is detected from an image that has been subjected to preprocessing using a detection model, and information on the detected number of people and the position is output.
Hereinafter, the processing of each stage will be described in detail.

(S1) Prior setting (steps S101 to S104)
FIG. 5 is a diagram showing the flow of the advance setting. In the preliminary setting, various conditions for performing person detection are input and set.

  In step S101, camera parameters 44 are set. The camera parameters 44 are manually input based on the specifications and installation information of the monitoring camera 4. Alternatively, automatic input by the camera parameter estimating means is also possible.

  FIG. 6 is a diagram showing a setting example of the camera parameters 44. The setting items include the installation height of the camera, the depression angle, the roll angle, the focal length, the horizontal size Cx of the image sensor, the vertical size Cy of the image sensor, the horizontal width Wcam of the camera image, the vertical width Hcam of the camera image, and the like.

  In step S102, a learned model to be used in human head detection is specified as the detection model 42. The trained model is a model in which a neural network such as a CNN is adopted, and the structural parameters of the network are calculated using an image including a large amount of a human head of the same size.

In step S103, the detection model attribute 43 is specified.
FIG. 7 is a diagram illustrating a data structure of the detection model attribute 43. Items of the detection model attribute 43 include a recommended image width Wopt, a recommended image vertical width Hopt, and a recommended head size Vopt (width × height) in the detection model 42.

  The recommended image sizes Wopt and Hopt specify the size of the input image that the detection model 42 can accept. If the size of the input image is different from the recommended image size, the size of the input image is changed (reduced / enlarged).

  The recommended head size Vopt is the size of a head image that can be detected as a head by the detection model 42. In a camera image, a person far from the camera may not be detected because the size of the head on the screen is reduced. Therefore, the recommended head size Vopt is specified based on the head size at the time of learning of the detection model 42. For example, the average size of the head image in the learning data is designated as the recommended head size.

  In step S104, the detection area is set by the detection area setting unit 35. In the camera image, an area where a human head is to be detected is set.

  FIG. 8 is a diagram showing a screen for setting a detection area. An image from the monitoring camera 4 is displayed on a screen 70 of the video display device 7. Here, a road 71 and a person 72 are displayed. In setting the detection area, the user operates the mouse or the like to receive the operation input unit 39, and connects four positions on the screen to generate one polygon. An area in the polygon is set as a detection area, and its boundary is displayed on the screen. In the example of FIG. 8, the area of the polygon ABCD (shown by a broken line) is the detection area 73, and the person 72 included therein is the detection target. Note that a plurality of detection regions can be set by performing the above operation a plurality of times. By pressing the reset button 78, the user can reset the set area and reset it. By pressing the save button 79, the set detection area 73 is stored as the detection area information 45.

  FIG. 9 is a diagram showing the data structure of the detection area information 45. In the detection area information 45, the area ID is a number for distinguishing each polygon when there are a plurality of polygons. The polygon coordinates describe a plurality of (four points) on-screen coordinates constituting the polygon.

(S2) Generation of pre-processing parameters (steps S201 to S213)
FIG. 10 is a diagram showing a flow of the preprocessing parameter generation. The generation of the preprocessing parameters is performed mainly by the on-screen head size calculation unit 36 and the preprocessing parameter calculation unit 37. Here, the size of the head image displayed on the screen in the detection area is calculated, and the preprocessing parameters are generated based on the calculated size.

  In step S201, the on-screen head size calculation unit 36 reads the camera parameters 44 from the camera parameter DB 16.

In step S202, the relationship between the coordinates of the person and the size of the head is calculated on the assumption that the person stands on the flat ground at all positions on the screen. In calculating the size of a person's head, for example, the following preconditions are used, and the values of these preconditions change depending on the usage environment.
-The person's height is 160 cm.
-The human head is a sphere with a diameter of 40 cm.
・ The person's activity range is on flat ground.

  The specific method of calculating the head size assumes that the surveillance camera 4 is regarded as a pinhole camera, and that the center of the prerequisite human head is present at an arbitrary pixel position on the screen using the camera parameters 44. Then, the world coordinates of the person's head are calculated. Further, the size (width and height) of the human head on the screen is calculated from the world coordinates of the human head. As a result of the calculation, an on-screen head size list is generated.

  FIG. 11 is a diagram showing the data structure of the head size list on the screen. The on-screen head size list 80 includes on-screen coordinates and head size. It is assumed that all image points within the size of the input image from the camera (camera image width Wcam, image height Hcam in FIG. 6) are used as the coordinates on the screen. When a person stands at each image point, the size (horizontal and vertical) of the displayed head is described by the number of pixels. As can be seen from FIG. 11, the head size is calculated to be small at coordinates on the screen far from the camera, and the head size is calculated large at coordinates on the screen near the camera.

  In step S203, the head size in the detection area is extracted using the detection area information 45 set by the user. That is, in step S202, the head size in the entire region of the camera image has been calculated, but the head size in the detection region is now narrowed down.

Next, the preprocessing parameter calculation unit 37 calculates a preprocessing parameter 46 used by the preprocessing unit 32 using the information on the head size in the detection area.
In step S204, the corresponding detection model attribute 43 is read from the detection model attribute DB 15.

In step S205, the image reduction ratio is calculated using the recommended image width Wopt and the recommended image height Hopt of the detection model attribute 43, and the camera image width Wcam and the camera image height Hcam of the camera parameters 44.
Horizontal reduction ratio = Recommended image width Wopt / camera image width Wcam
Vertical reduction ratio = Recommended image vertical width Hopt / camera image vertical width Hcam
If the camera image size is smaller than the recommended image size, the reduction ratio becomes larger than 1 and the image is enlarged. Here, the case where the image is reduced (reduction ratio <1) will be described.

  In step 206, the head size in the reduced image is calculated from the head size of the detection area using the image reduction rate, and the minimum head size Vmin in the detection area is calculated. Generally, in a camera image, the head size is smaller in the depth direction. For example, in the case of FIG. 8, since the CD straight line is located at the deepest position in the detection area 73, the reduced head size at the points C and D is set to the minimum head size Vmin.

  In step S207, an ideal enlargement ratio Gopt for enlarging the minimum head size Vmin obtained in step S206 to the recommended head size Vopt specified in the detection model attribute 43 is calculated. That is, Gopt = Vopt / Vmin. Note that the enlargement ratio Gopt is a value at the CD position of the detection area 73, and the enlargement ratios at other positions are smaller than this and change linearly.

  In step S208, with respect to the ideal enlargement ratio Gopt obtained in step S207, the enlargement ratio is corrected so that the image distortion does not exceed a certain threshold. This is because the detection accuracy decreases as the image distortion increases. Specifically, in the case of a person's head, the enlargement ratio is corrected so that the ratio of the upper width to the lower width in the shape of the enlarged head becomes a certain threshold, for example, 1.2 or less. The corrected enlargement ratio is defined as an effective enlargement ratio Geff. If the image distortion is equal to or less than a certain threshold, the ideal enlargement ratio Gopt becomes the effective enlargement ratio Geff as it is.

  In step S209, in the case of the detection area 73 in FIG. 8, a rectangle including the polygon ABCD is set as a cutout image, and coordinates of points C and D in the cutout image are calculated. Here, the points C and D are vertex coordinates when the image is enlarged (projective transformation), and will be referred to as reference points.

  In step S210, the coordinates of the reference points C 'and D' (shown in FIG. 13) after the enlargement at the effective enlargement ratio Geff are calculated for the reference points C and D. The points A and B are fixed points in the image enlargement, and the coordinates do not change.

  In step S211, projection transformation parameters are calculated using the coordinates of each point of the polygon ABCD and the polygon ABC'D 'after the enlargement. The method of calculating the parameters of the projective transformation is as follows.

  In Equation 1, x and y are coordinates before enlargement, u and v are coordinates after enlargement, and a, b, c, d, e, f, g, and h are conversion coefficients. When the values of the coordinates (x, y) of the four points of ABCD and the coordinates (u, v) of the four points of ABC'D 'after the enlargement are substituted into Equation 1, eight conversion equations are obtained. From these, eight transform coefficients a to h are obtained, and these are described in a matrix form of the following Expression 2 to be the projective transformation parameters.

In step S212, a pre-processing parameter 46 is generated using the projective transformation parameter.
FIG. 12 is a diagram illustrating a data structure of the pre-processing parameter 46. The items of the pre-processing parameter 46 include a cut-out area, an image reduction ratio, a projection conversion parameter, and an output image size. The cutout area is a rectangular area (ABC'D 'in FIG. 13) of the detection area 73 that includes the polygon ABCD, and is indicated by the coordinate values of its diagonal points. The image reduction ratio is the reduction ratio in the horizontal direction and the vertical direction calculated in step S205. The projective transformation parameters are the values calculated in step S211 and represent the transformation coefficients a to h in a matrix. The output image size is the size after image reduction and has the same value as the recommended image horizontal width Wopt and the recommended image vertical width Hopt of the detection model attribute 43.

In step S213, the user performs a pre-processing parameter confirmation operation.
FIG. 13 is a diagram illustrating a confirmation screen of a preprocessing parameter. On the screen 70 of the video display device 7, a preview of the effect of the pre-processing is displayed together with the camera image. That is, the currently set detection area 73 (ABCD, indicated by a broken line) is displayed, and the image area 83 (ABC'D ', indicated by a dashed line) cut out in the pre-processing, and the detection can be detected by projective transformation. An area 85 (indicated by a dot pattern) is displayed. Here, the reason why all of the detection areas 73 do not become the detectable areas 85 is that an area where image distortion exceeds a threshold is generated by the projective transformation.

  The user confirms this result, and if there is no problem, presses the save button 89, so that the pre-processing parameter 46 is stored in the pre-processing parameter DB 18. If the user wants to change the detection area 73, the user returns to the detection area setting (step S104) in FIG. 5 by pressing the reset button 88. Then, it is possible to change the detection area and calculate the pre-processing parameters again.

(S3) Pre-processing (Steps S301-S305)
FIG. 14 is a diagram showing the flow of the pre-processing. When the image input unit 31 sends a camera image to the preprocessing unit 32, the preprocessing unit 32 performs preprocessing on the camera image.

  In step S301, the detection area information 45 (detection area coordinates) is read from the detection area setting DB17. In FIG. 13, a detection area 73 indicated by polygon ABCD corresponds.

  In step S302, the pre-processing parameters 46 are read from the pre-processing parameter DB 18.

  In step S303, the image of the region to be detected is cut out using the information of the cutout region of the preprocessing parameter 46. In FIG. 13, a rectangular area 83 indicated by polygon ABC'D 'is cut out.

  In step S304, projective transformation is performed on the cut-out image using the projective transformation parameter of the preprocessing parameter 46. In the projective transformation, each pixel position in the image is transformed by using the above equation (2). In FIG. 13, the images of points C and D are enlarged and converted to the positions of points C 'and D'. By performing the projective transformation, it is possible to enlarge the head image of the person in the back of the shooting space.

  In step S305, the image after the projective transformation is reduced according to the image reduction ratio of the preprocessing parameter 46. Thus, the pre-processing ends.

(S4) Number of people measurement (steps S401-S411)
FIG. 15 is a diagram showing the flow of the number-of-people measurement process. When the pre-processing unit 32 sends the pre-processed image to the feature amount extraction unit 33, the process proceeds to person detection and number measurement processing. Here, feature amount extraction by the feature amount extraction unit 33, detection of a human head by the human head detection unit 34, and human position integration processing by the detection result integration unit 38 are performed.

  In step S401, the feature amount extraction unit 33 extracts a feature amount from an image. In the case of a CNN network, an image feature amount is extracted by a deep learning method such as a convolution operation or a pooling layer.

  In step S402, the human head detection unit 34 calculates the presence / absence of a human head, the center coordinates of the human head, and the head size using the extracted feature amounts. As a result of the human head detection, a list including the coordinate information of the human head is generated.

  However, the coordinate information generated here is coordinate information after preprocessing, and is different from the coordinates of the image input by the image input unit 31. Therefore, in step S403, the detection result integration unit 38 performs a person position integration process of converting the coordinates of the input image before the preprocessing.

The person position integration processing shown in step S403 includes steps S404 to S409.
In step S404, the pre-processing parameters 46 are read from the pre-processing parameter DB 18.

In step S405, the detection area information 45 is read from the detection area setting DB17.
In step S406, the camera parameters 44 are read from the camera parameter DB 16.

In step S407, using the image reduction ratio of the pre-processing parameter 46, the human head coordinate information generated in S402 is converted into coordinates before reduction.
In step S408, inverse projective transformation is performed to transform the human head coordinate information into coordinates before projective transformation. What is necessary is just to use the transpose of the projective transformation parameter (Formula 2) of the pre-processing parameter 46 as the parameter at the time of inverse projective transformation.

In step S409, using the information of the cut-out area of the preprocessing parameter 46, the coordinates of the person's head in the image before the cut-out are converted.
As described above, the person position integration processing is completed, and information on the person position described by the coordinates before the pre-processing is sent to the output control unit 40.

In step S410, the output control unit 40 outputs the person detection information to the video display device 7, and the result of the person detection is displayed.
FIG. 16 is a diagram illustrating a display example of a person detection result. The screen 70 of the video display device 7 displays the set detection area 73 together with the image from the monitoring camera 4. Also, based on the current person detection result, a rectangular detection mark 90 is displayed over the detected person's head position, and the latest detected number of people is displayed in the detected number of people column 91.

In step S411, the output control unit 40 outputs the person detection information to the number history storage device 8, and the person detection results are stored.
FIG. 17 is a diagram illustrating a data structure of a person detection result. The person detection result 92 describes the number of person heads at each detection time in the detection area 73, that is, the number of people. Further, in addition to the number of people, the coordinates of the detected person and the coordinates of the head may be described.

  As described above, the operation of the human head detection device 6 of the present embodiment has been described. When detecting a person in a predetermined area, by performing appropriate preprocessing including projection transformation of a captured image using camera parameters, The effect of improving the detection accuracy of the person, and hence the accuracy of the measurement of the number of persons, while suppressing an increase in the cost of the detection process can be obtained. That is, in the present embodiment, the processing load at the time of detection is suppressed by performing the projective transformation so that the image of the person in the back as viewed from the camera in the detection area is larger than the image in the foreground. It is possible to improve detection accuracy including a person in the back. Further, by reducing the size of the input image to the recommended image size in the pre-processing, the processing load of human detection by the convolutional neural network can be reduced.

  In the above-described embodiment, the head of a person has been described as an example of an object to be detected. However, it is needless to say that an arbitrary object can be detected by using a learned detection model. In addition, the number of monitoring cameras 4 may be plural, and furthermore, a plurality of human head detecting devices 6 and the number of people history accumulating devices 8 are provided, and the number of persons is measured while cooperating between the devices. Is also good.

  The present invention is not limited to the above embodiment, and includes various modifications. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to one having all the described configurations. Further, for a part of the configuration of the embodiment, it is also possible to add / delete / replace another configuration.

  In addition, each of the above-described configurations, functions, processing units, and the like may be partially or entirely realized by hardware, for example, by designing an integrated circuit. Further, the technical elements of the above-described embodiments may be applied independently, or may be applied by being divided into a plurality of parts such as a program component and a hardware component.

  1: person counting system, 3:72: person, 4: surveillance camera, 6: person head detecting device, 7: video display device, 8: person history storage device, 11: CPU, 12: storage unit, 32: before Processing unit, 33: feature amount extraction unit, 34: human head detection unit, 35: detection area setting unit, 36: on-screen head size calculation unit, 37: pre-processing parameter calculation unit, 38: detection result integration unit, 40: output control unit, 42: detection model, 43: detection model attribute, 44: camera parameter, 45: detection area information, 46: pre-processing parameter, 73: detection area, 80: screen head size list, 83: Cutout image area, 85: detectable area, 90: detection mark, 91: number of detected persons column, 92: result of person detection.

Claims (8)

In an object detection device that detects a predetermined object from an image captured by a camera,
A detection region setting unit that sets a detection region for detecting the object with respect to the input image,
A preprocessing unit that performs preprocessing including projective transformation on the input image,
A preprocessing parameter calculation unit that generates a parameter used in the preprocessing,
An object detection unit that detects the object from the image that has been subjected to the preprocessing using a predetermined detection model,
The pre-processing parameter calculation unit, the size of the object on the image calculated from camera parameters at the time of shooting of the camera, assuming the object has a fixed size, and the object detectable using the detection model An object detection device that generates the parameters of the preprocessing including the projective transformation based on a recommended size on the image of (1). The object detection device according to claim 1,
The pre-processing parameter calculation unit, the projective transformation so that the minimum value of the size of the object calculated from the camera parameters in the detection area is equal to or more than the recommended size of the object that can be detected using the detection model. An object detection device, which calculates the following parameters: The object detection device according to claim 2,
The pre-processing parameter calculation unit may calculate the projective transformation parameters such that an enlargement ratio of an image located in the back as viewed from the camera is larger than an image located in the foreground in the detection area. Object detection device. The object detection device according to claim 3,
The object detection method, wherein the preprocessing parameter calculation unit corrects the parameters of the projective transformation so that distortion of the shape of the object after the enlargement does not exceed a threshold when the image is enlarged by the projective transformation. apparatus. The object detection device according to claim 1,
The pre-processing further includes an image cutout from the input image based on the detection area, and a process of reducing the input image to a recommended image size that allows the detection model to be accepted. Object detection device. The object detection device according to claim 5,
The detection result of the object by the object detection unit, a detection result integration unit that converts the position of the object detected based on the parameters of the preprocessing into coordinates of the input image and integrates them,
An output control unit that outputs the integrated object detection information to an external device,
An object detection device comprising: In an object detection method for detecting a predetermined object from an image captured by a camera,
Setting a detection area for detecting the object in the input image,
Performing preprocessing including projective transformation on the input image;
Calculating a pre-processing parameter to generate a parameter used in the pre-processing,
Detecting the object from the pre-processed image using a predetermined detection model,
In the pre-processing parameter calculation step, the size of the object on the image of the object is calculated from camera parameters at the time of shooting by the camera, assuming that the object has a fixed size, and the object detectable using the detection model. Generating a parameter for the preprocessing including the projective transformation in comparison with a recommended size on the image. The object detection method according to claim 7, wherein
The pre-processing step further includes an image cutout from the input image based on the detection region, and a process of reducing the input image to a recommended image size that allows the detection model to be accepted,
In the step of detecting the object, an object is detected by using a convolutional neural network for the image on which the preprocessing has been performed.

Images