JP2020013206A - Device for detecting two-wheeled vehicle from moving image/camera, program, and system - Google Patents

Abstract

To provide a device for accurately recognizing even a moving object such as a bicycle or a motorcycle or the like which appears very small.SOLUTION: A device for detecting a two-wheeled vehicle preliminarily designates a range in which dynamics are detected in a video obtained from a camera as a maximum area, and performs dynamics detection within the maximum area. By reducing/enlarging an area detected by the dynamics detection to a size suitable for an image recognition model such as a convolutional neural network learned by deep learning, a small object such a pedestrian, a bicycle, a motorcycle, or the like which appears small in the video is accurately identified.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an apparatus, a program, and a system (hereinafter, referred to as “apparatus or the like”) for automatically detecting a motorcycle or a motorcycle from a camera. Specifically, from a video from a camera, a moving image stored in a storage device, and a still image file, a motorcycle such as a bicycle or a motorcycle on which a person rides is determined using a motorcycle detection model generated in advance based on deep learning. In addition, the present invention relates to a motorcycle detection device and the like having improved accuracy in detecting a person and a motorcycle or a motorcycle.

  In recent years, with the improvement of image recognition technology, moving objects are detected from moving images and still images (hereinafter, simply referred to as “images”) taken by a camera mounted on a vehicle, and walking with a bicycle or a motorcycle and their occupants is performed. 2. Description of the Related Art A technique for detecting a moving object in an image from a camera using an image recognition technology such as an image recognition device (Patent Document 1) for recognizing a person has been actively developed.

  Another conventional method of detecting a moving object from an image captured by a camera is to detect an object such as a pedestrian or a bicycle by using an image obtained from a stereo camera attached to a pillar or the like on a road. Then, when the presence of a pedestrian or the like is recognized based on image data captured by a device for estimating motion information such as the position and speed of a target object (Patent Document 2) or an imaging device for capturing an image of the outside of a vehicle. There is an obstacle detection system (Patent Literature 3) and the like that can notify the driver of another vehicle of the presence of a pedestrian or the like as a recognition target object.

Japanese Patent No. 4669344 Japanese Patent No. 5075672 JP 2006-324423 A

  The method of detecting moving objects (pedestrians, automobiles, two-wheeled vehicles, etc.) used in these conventional examples can also be applied to video taken by telephone poles on the roadside or cameras installed in shopping streets. By counting the number of moving objects such as pedestrians, bicycles, and automobiles detected within a certain period, it is expected that traffic on a road, traffic on a shopping street, and the like can be calculated.

  However, cameras installed at shopping streets and intersections sometimes take pictures with the shooting location subtracted, that is, a bird's-eye view, in which case, pedestrians, bicycles, motorcycles, etc. Since the moving object such as a two-wheeled vehicle is very small, the moving object detection method of the above-described conventional examples (particularly, Patent Documents 1 and 3) which targets a relatively large moving object photographed by a vehicle-mounted camera. Thus, there has been a problem that it is not possible to accurately recognize whether the moving object in the video is a pedestrian or a two-wheeled vehicle such as a bicycle or a motorcycle.

  In a conventional example using a stereo camera (Patent Document 2), a relatively small target can be detected by defining a monitoring region and detecting a target region existing in the monitoring region. However, the method is intended for an image using a stereo camera, and cannot be directly applied to an image captured by a normal one-lens camera.

  In order to solve such a problem, it is conceivable to replace equipment such as a camera installed at an intersection or a shopping mall with new equipment such as a stereo camera, but this is not practical because it requires a cost for equipment investment. Therefore, there is a need for a mechanism capable of detecting the dynamics in a video and recognizing what it is by utilizing the existing facilities without replacing existing facilities such as cameras installed at an intersection or a shopping street.

  Therefore, in the present invention, in an image obtained from a camera, an area detected by dynamic detection is set to a size suitable for an object detection model (image recognition model) such as a convolutional neural network (CNN) learned by deep learning. A device or the like that can accurately identify even small objects such as pedestrians, bicycles, and motorcycles that appear small in an image by performing reduction and enlargement as much as possible.

The motorcycle detection device according to one embodiment of the present invention includes:
A motion detection unit that extracts a detection area that detects a motion from video data including a moving image or a continuous still image captured by a camera, and data corresponding to the detection area are input to an object detection model, and an object in the detection area is input. And a motorcycle detecting unit for detecting
The dynamic state detection unit performs image processing of enlargement or reduction in order to reduce data corresponding to the detection area to a preset size,
The object detected by the motorcycle detection unit includes at least a motorcycle including a bicycle and a motorcycle.

As a preferred embodiment of the motorcycle detection device according to the present invention,
When a motorcycle including at least a bicycle and a motorcycle is detected from the detection area by the object detection model, an identifier is assigned to the detection area.

As a preferred embodiment of the motorcycle detection device according to the present invention,
The identifier, the detection area extracted this time, before the detection area is extracted, comparing the previous detection area was extracted, within the range of a preset individual identification time, When the distance is within the range of the individual identification distance set in advance, the identifier is the same as the previous detection area, and otherwise, the identifier is a new identifier.

As a preferred embodiment of the motorcycle detection device according to the present invention,
The detection area is a difference area data within a predetermined detection maximum area and a range equal to or more than a predetermined detection minimum area,
The difference area data is characterized in that the difference area data is data having an area inside the contour of difference data obtained by extracting a difference between a frame of the video data and a subsequent frame.

As a preferred embodiment of the motorcycle detection device according to the present invention,
The differential data is processed into a smooth image by image processing of expansion or contraction.

As a preferred embodiment of the motorcycle detection device according to the present invention,
The detection area is characterized in that an image is analyzed and brightness and contrast are adjusted.

As a preferred embodiment of the motorcycle detection device according to the present invention,
The detection area is stored in the detection database as a record in which, in addition to the ID corresponding to the identifier, a detection time, a type representing a bicycle or a motorcycle, coordinates of a center point of the detection area, and an area of the detection area are recorded. Features.

As a preferred embodiment of the motorcycle detection device according to the present invention,
In the detection database, a traveling direction of a bicycle or a motorcycle represented by the type is determined based on a locus of movement of the center point coordinates from an old record at the detection time to a new record at the detection time for each ID. It is characterized by doing.

As a preferred embodiment of the motorcycle detection device according to the present invention,
In the detection database, from a plurality of records specified by the ID, a center point coordinate in a record recorded first in time series is set as a start point, and a center point coordinate in a record recorded last is set as an end point. And the end point, perform a projective transformation based on a preset projective transformation coordinate,
Calculate an estimated distance based on the distance between pixels set in advance from the coordinates of the start point and the end point after the projective transformation, and the detection time is recorded in the record including the start point and the record including the end point. From the detected time, the travel time is calculated, and the passing speed of the bicycle or motorcycle represented by the type is estimated,
The predetermined projective transformation coordinates are coordinates obtained by projecting a scene taken by the camera as viewed from directly above.

As a preferred embodiment of the motorcycle detection device according to the present invention,
By comparing the passing speed of the bicycle or motorcycle represented by the type with a preset boarding / disembarking determination speed, to determine the boarding / disembarking state,
If the passing speed is equal to or higher than a preset boarding / disembarking determination speed, the boarding / disembarking state indicates that the vehicle is boarding,
If the speed is lower than the predetermined boarding / disembarking determination speed, the boarding / disembarking state indicates that the vehicle is getting off.

As a preferred embodiment of the motorcycle detection device according to the present invention,
Based on the traveling direction, the passing speed, and the boarding / disembarking state, at least one value of the number of traffic per unit time per direction, the boarding / dismounting ratio by direction, and the traveling speed by direction is calculated. .

As a preferred embodiment of the motorcycle detection device according to the present invention,
The object detection model is a convolutional neural network (CNN).

  A two-wheeled vehicle detection program according to one embodiment of the present invention is characterized by being executed by a computer to cause the computer to function as the two-wheeled vehicle detection device.

  A motorcycle detection system according to one embodiment of the present invention includes a storage device that acquires and stores moving image or continuous still image data from a camera, and a computer that is connected to the storage device via a network and executes the program. And characterized in that:

  By using the motorcycle detection device, method and program according to the present invention, equipment such as a camera installed at an intersection or a shopping mall is not replaced with new equipment, and the existing equipment is used as it is to change the dynamics in the video. By detecting, it is possible to recognize whether the detected object is a person or a two-wheeled vehicle such as a bicycle or a motorcycle.

  In addition, by applying the present invention to video data obtained from cameras installed at intersections and shopping streets, the number of traffic of bicycles and motorcycles in each direction per unit time, the ratio of riding and getting off by direction, The amount of traffic of the motorcycle at an intersection or a shopping street such as a traffic speed can be grasped for each time zone, whereby the monitoring of the motorcycle traveling at the intersection or the shopping street can be automated.

It is a figure showing the composition outline of the device etc. in the motorcycle detection system concerning one embodiment of the present invention. It is a figure showing composition of a movement detection analysis program for making a computer concerning one embodiment of the present invention function as a motorcycle detection device. FIG. 4 is a diagram illustrating an example of projective transformation of coordinates of video data. It is a flowchart of the process which detects a dynamic from video data and specifies a detection area. FIG. 4 is a diagram illustrating a state of processing from detecting dynamics from video data to specifying a detection area. It is a flowchart of a process of assigning an ID to each of the detection areas, tracking the movement of the detection area for each ID in time series, and storing it as detection data in a detection database. It is the schematic of the part set as the detection area and the part set as the exclusion data in video data. It is a still image which shows the example of the result of dynamic detection by the two-wheel detection apparatus which concerns on one Embodiment of this invention, and the example of the result of dynamic detection only by deep learning. It is a still image which shows the example of the result of dynamic detection by the two-wheel detection apparatus which concerns on one Embodiment of this invention, and the example of the result of dynamic detection only by deep learning.

  An embodiment of the present invention will be described below with reference to the drawings. In all the drawings for describing the embodiments, the same portions are denoted by the same reference numerals in principle, and the repeated description thereof will be omitted.

  FIG. 1 shows a schematic configuration of devices and the like in a motorcycle detection system according to an embodiment of the present invention. The motorcycle detection system of the present invention includes a motorcycle detection device 10 and a camera 20 such as a security camera, and video data stored in a storage device 30 of the motorcycle detection device 10 is captured by the camera 20 and stored in a storage device 40. Video data. For example, a moving image or a continuous still image (hereinafter, simply referred to as “video”) captured by the camera 20 is temporarily stored in the storage device 40 as video data and transmitted to the motorcycle detection device 10 via a network. The data is stored in the storage device 30.

  In the embodiment shown in FIG. 1, the computer functions as the motorcycle detection device 10 by causing the computer to execute the movement detection analysis program 50, but is not limited thereto. It is also possible to implement at least a part or all of the functions in hardware by an application specific integrated circuit (ASIC) or the like.

  The dynamic detection analysis program 50 mainly has two functions, that is, dynamic detection by image processing and motorcycle detection by deep learning. In addition to motorcycles such as bicycles and motorcycles in the video, movement of people, automobiles, and the like is performed. Objects can be detected and their moving speed, moving direction, traffic amount, and the like can be measured. The motion detection is performed on the entire image (that is, the entire screen on which the image is projected). For the detected area, a detection maximum area and a detection minimum area described later are used as noise removal. A more specific configuration of each function of the movement detection analysis program 50 is as shown in FIG.

  FIG. 2 shows a configuration of a movement detection analysis program for causing a computer according to an embodiment of the present invention to function as a motorcycle detection device. The movement detection analysis program 50 that causes a computer to function as the motorcycle detection device 10 includes a movement detection unit 51 based on image processing and a motorcycle detection unit 53 based on deep learning. The movement detection unit 51 and the motorcycle detection unit 53 can be realized on a computer that has executed the movement detection analysis program 50.

  For example, an image processing library OpenCV which is existing open source software (OSS) is used for the dynamic detection unit 51 by image processing, and the existing object detection algorithm SSD: Single is used for the motorcycle detection unit 53 by deep learning. A Shot MultiBox Detector can be used. For the deep learning, for example, an existing deep learning framework, TensorFlow / Keras, can be used. However, the deep learning is not limited to the above-described framework, and it is not necessary to specify a neural network or a recognition algorithm, and any method can be used as long as it can recognize an object from a general image. Further, in the embodiment shown in FIG. 2, the motorcycle detection unit of the present invention is referred to as “the motorcycle detection unit 53 based on deep learning”. However, the present invention is not limited to this, and the object detection generated by a method other than deep learning is not limited thereto. Models can also be used.

  The user can make the following various pre-settings for the movement detection analysis program 50.

(1) Maximum detection area, minimum detection area:
The user can set a “maximum detection area” and a “minimum detection area” to determine whether an area detected as the dynamics of video data is to be processed. Specifically, the maximum detection area and the minimum detection area are designated by area or rectangular coordinates. The detection maximum area and the detection minimum area are used to determine whether an area detected as dynamic is targeted or excluded.

(2) Individual identification time:
The user can set a time for identifying an individual such as “whether the same bicycle” or “the same motorcycle” between frames as an “individual identification time” for an area detected by dynamic detection. . The dynamic detection analysis program 50 sets an ID (identifier) as the same individual in a detection area that is equal to or less than the set value (second) and within the “distance for individual identification”.

(3) Individual identification distance:
The user can set a distance for identifying an individual such as “whether the same bicycle” or “same motorcycle” between frames as an “individual identification distance” in an area detected by dynamic detection. . The movement detection analysis program 50 sets an ID (identifier) as the same individual in a detection area equal to or less than the set value (pixel) and within the “individual identification time”.

(4) Boarding / alighting judgment speed:
The user can set the "ride / get-off determination speed" used to determine the getting-on / off state of the motorcycle in the detection area. For example, the riding state can be set to 6 km / h or more.

(5) Projected transformation coordinates:
The user can set the coordinates for projective transformation of the image (four-point coordinates of the transformation source starting point and four-point coordinates of the transformation destination origin) for calculating the speed as “projection transformation coordinates”. FIG. 3 shows an example in which the coordinates of the video data are projected and transformed. In the example shown in FIG. 3, the user can set a conversion source start point and a conversion destination start point. For example, the coordinates of the four corners of the cross point of the cross shown in the video image are set as the conversion start points, and the position (coordinates) where the cross point of the cross can be seen from directly above is set as the conversion start point. The detection trajectory of the object such as a motorcycle before the projective transformation is seen from the front to the back of the intersection or from the back to the front, but after the projective transformation, the intersection of the cross is transformed as viewed from directly above, and the detected trajectory is detected. Is converted from bottom to top or top to bottom.

(6) Distance between pixels:
The user can set the distance between the pixels of the image after performing the projective transformation for calculating the speed as the “distance between pixels”.

(7) Deep learning input image size:
The user can set the size of the image input to the motorcycle detection unit 53 by the deep learning as the “deep learning input image size”. The dynamic detection analysis program 50 can enlarge or reduce the image of the detection area in accordance with this size to obtain a deep learning input image. In other words, the area (detection area) detected and cut out as the movement by the movement detection becomes the input image of the deep learning, and the input image size can be adjusted according to this setting at the time of the input.

(8) Deep learning input image processing:
The user can set the setting for adjusting the contrast and the contrast of the image in the detection area as “deep learning input image processing”. For example, when adjusting luminance, a threshold value for performing correction and a gamma correction value are set.

  The dynamic detection analysis program 50 performs dynamic detection from the video data, identifies a bicycle / motorcycle by deep learning for the detected dynamic detection target, creates “detection data”, and sets the detection data as one record. Store in the detection database. The procedure for creating “detection data” from video data is shown in the flowcharts of FIGS.

  FIG. 4 shows a flowchart of a process of detecting a dynamic from video data and specifying a detection area. FIG. 5 shows a state of processing from detecting dynamics from video data to specifying a detection area. Referring to FIGS. 4 and 5, the procedure up to specifying the detection area is as follows. First, the current frame and the subsequent frame of the video data are compared, and a difference between the frames is extracted (step S100). Next, “difference data” is created by binarizing the extracted difference between black and white (step S101). FIG. 5A shows a process of extracting the difference between the current frame and the subsequent frame and binarizing the difference between black and white. The portion where the current frame and the subsequent frame overlap is white, and the portion where it does not overlap is black.

  It should be noted that the user can set the frame comparison interval as a “comparison frame interval” in advance. Such pre-setting is for responding to the characteristics of the image (“movement is severe”, “there is little movement”, etc.) or the processing performance of PC hardware depending on the number of connected cameras.

  Normally, since the “difference data” includes fine noise and the like, the image processing of expansion and contraction is performed to process the image into a smooth image (step S102). FIG. 5B shows a state where the “difference data” is processed into a smooth image by the expansion / reduction processing.

  The “difference data” that has undergone the image processing of expansion and contraction is converted into “difference area data” in which the outline is extracted by image processing and the inside of the outline is grouped together (step S103). In FIG. 5C, the image is converted into “difference area data” in which the inside (filled area) of the outline of the “difference data” after processing into a smooth image is one area.

  It is determined whether the “difference area data” is within the ranges of the “detection maximum area” and “minimum detection area” set in advance (step S104), and if it is within the range (YES in step S104), The “difference area data” is set as the “detection area” (step S105), and if it is out of the range (NO in step S104), the “difference area data” is set as the exclusion data (step S106). Here, FIG. 7 schematically shows a part that is set as a detection area and a part that is set as exclusion data in video data. In the difference area data, the outer periphery has a maximum detection area (in the example of FIG. 7, the outermost rectangular area of the detection area) and a minimum detection area (in the example of FIG. 7, the innermost rectangular area of the detection area). ) Is set as a detection area.

  “Difference area data” larger than the preset “detection maximum area” is set as the exclusion data, and “difference area data” smaller than the preset “detection minimum area” is also set as the exclusion data. .

  FIG. 6 shows a procedure for generating “detection data” based on the “detection area”. First, image processing for enlargement / reduction of the “detection area” to the size of the “deep learning input image size” specified in advance is performed (step S110), and “deep learning input image processing” is performed on the “detection area”. If there is a setting, the image is analyzed and the brightness and contrast are adjusted (step S111). If the “deep learning input image processing” is not set, the brightness and contrast are left as they are and no adjustment is made.

  Then, the object detection is performed on the “detection area” on which the image processing has been performed, using the object detection model that has been learned by the deep learning (step S112). By the object detection of the deep learning, it is determined whether or not the vehicle is a “bicycle” or a “motorcycle” (step S113). If it is detected as a “bicycle” or “motorcycle” motorcycle (YES in step S113), it is set as a “detection area” to which an ID is assigned (step S114), and if it is not detected as a “bicycle” or “motorcycle” motorcycle. (NO in step S113), the processing for detecting the motorcycle ends. In addition, even when a motorcycle is not detected, when a person or a car is detected, an ID for identifying them may be assigned.

  The “detection area” to which the ID is assigned is compared with the information of the “detection area” detected last time, and the distance of the center point (center point coordinates) within the “individual identification time” and the “individual identification distance” Is determined (step S115). The “detection area” to which the ID is assigned is compared with the information of the “detection area” detected last time, and the distance of the center point (center point coordinates) within the “individual identification time” and the “individual identification distance” ("YES" in step S115), the same ID as the ID of the previously detected "detection area" is set in the "detection area" (step S116).

  On the other hand, the “detection area” to which the ID is assigned is compared with the information of the “detection area” detected last time, and the distance of the center point (center point coordinates) within the “individual identification time” is “individual identification”. If it is not within the “distance” (NO in step S115), a new ID is set in the “detection area” (step S117). When a plurality of “detection areas” exist, different IDs are assigned to the respective “detection areas”.

The “detection area” for which the ID has been set is stored as “detection data” (step S118). “Detection data” corresponds to one record and includes the following information.
<Time>, <ID>, <type: bicycle, motorcycle>, <center point coordinates X, Y>, (<area>)
<Time> is the time at which the detection area was detected, and ID is an identifier corresponding to the identifier (ID) assigned to the detection area. The <type> indicates whether the individual (object) in the detection area is a bicycle or a motorcycle, and the <center point coordinates> are coordinates of the center point of the detection area. <Area> is the area of the detection area, and the information on the area may not be included in the detection data. The detection data is stored in the detection database as one record. When the video data obtained from the camera is stored as a moving image file, when analyzing the moving image file, <time> sets the time of the first frame, and sets the number of frames and the FPS (frame of 1 second). ) Can be calculated based on the difference from

“Traffic data” can be created for each ID from a plurality of “detection data” (a plurality of records) stored in the detection database. The motorcycle detection device 10 (the movement detection analysis program 50) can calculate a traffic amount or the like from “traffic data”. The record of “passing data” includes the following information.
<Time>, <ID>, <Type: Bicycle, Motorcycle>, <Progress direction>, <Passing speed>, <Ride / Get off>
<Time> is an arbitrary time (for example, the time recorded in the first record, the time recorded in the middle record, and the like) for each ID from the plurality of “detection data” (plural records) stored in the detection database. Or, the time recorded in the last record).

  <Progress direction> is determined using the coordinates of the center point recorded in each "detection data" (each record) for each ID from a plurality of "detection data" (a plurality of records) stored in the detection database. Is done. The type of the traveling direction is determined in advance (eg, north, south, east, west, left, right, etc.), and the direction is determined, for example, by determining the straight line of the center point of the last two records (or several records) according to the angle.

  The <passing speed> is, for each ID from a plurality of “detection data” (a plurality of records) stored in the detection database, a center point (start point) recorded in the first record and a center point recorded in the last record. For a point (end point), projective transformation is performed based on a predetermined “projection transformation coordinate” setting, a distance is calculated from the coordinates of the converted start point and end point, and a predetermined “inter-pixel distance” setting and start point are calculated. The passing speed can be obtained based on the time at the end point.

  <Get on / off> is information for determining whether the bicycle or motorcycle represented by the type is on or off. <Get on / off> compares the passing speed with a predetermined “get on / off judgment speed” setting. If it is fast, it is determined that the boarding state is set, and a value indicating the getting on state is set. And set a value indicating the getting-off state.

  The motorcycle detection device 10 (the dynamic detection analysis program 50) uses the generated “traffic data” to determine the time unit based on information such as <type>, <traveling direction>, <passing speed>, and <riding / dismounting>. , The calculation of the number of traffic by direction, the calculation of the boarding / disembarking ratio by direction, and the calculation of the traffic speed by direction can be obtained.

  The demonstration experiment used video data of a security camera set at a certain place at 12:00. The video data used was 640 × 480 pixels, 3 FPS and coarse image quality. In the motorcycle detection device 10 (dynamic detection analysis program 50) of the present invention, the number of detected motorcycles was 215. The visual count was 221. The two-wheeler detection device 10 (the dynamic detection analysis program 50) of the present invention was able to detect a two-wheeler such as a motorcycle or a bicycle almost as much as a human.

In contrast, when the framework of deep learning is simply used without performing dynamic detection, as shown in the following table, the number of detected frames is 12 and the number of detected motorcycles is 10. Was.

  8 and 9 are still images showing an example of a result of dynamic detection analysis by the two-wheel detection device according to an embodiment of the present invention and an example of a result of object detection by only deep learning. 8 and 9 are the same, but are taken from different angles. The video data was 1920 × 1080 pixels, 30 FPS, and a motorcycle was detected from a moving image captured by a smartphone camera.

  Comparing the images of FIGS. 8A and 8B, in the image recognition by the motorcycle detection device 10 (the dynamic detection analysis program 50) of the present invention, all persons appearing in the image are recognized as humans, and one vehicle is further recognized. Except for this, it can be seen that the person is aware of the bicycle on board. On the other hand, in image recognition using an object detection model based only on deep learning, not all persons could be recognized, and further, no bicycle could be recognized.

  9 (a) and 9 (b), the image recognition by the motorcycle detection device 10 (dynamic detection analysis program 50) of the present invention shows a small image of a person crossing a pedestrian crossing and the vicinity of a pedestrian crossing. It can be seen that the person is recognized as a human, and the bicycle on and near the pedestrian crossing is also recognized. On the other hand, image recognition using an object detection model based only on deep learning only barely recognizes the car that appears in the image, but cannot recognize everyone in the image, and even recognizes a single bicycle. I couldn't. It should be noted that the video data used for comparison in FIGS. 8 and 9 has higher image quality than the video of a normal security camera. When video data (for example, 640 × 480 pixels, 3FPS) of an actual security camera was used, nothing could be recognized by image recognition using an object detection model based only on deep learning.

  As described above, the motorcycle detection device 10 (the dynamic detection analysis program 50) according to one embodiment of the present invention uses an image based on an object detection model (image recognition model) such as a convolutional neural network (CNN) learned by deep learning. As preprocessing for recognition, the size of the detection area can be reduced or enlarged so as to be a size suitable for the object detection model. That is, the motorcycle detection device 10 (the movement detection analysis program 50) performs the movement detection by the movement detection by the image processing, and is suitable for extracting the area detected by the movement detection to the image recognition performance of the object detection model by the deep learning. Can be reduced or enlarged to a different size.

  As described above, the motorcycle detection device 10 (the dynamic detection analysis program 50) shrinks and enlarges the detection area so as to have a size suitable for the object detection model (image recognition model), so that the detection area appears small in the video. Even small objects such as pedestrians, bicycles and motorcycles can be accurately identified.

  As another demonstration experiment, in order to solve the dangerous act of riding while riding a bicycle in a pedestrian paradise at a shopping street, we visualize the traffic situation of bicycles from images of security cameras and consider solutions An experiment was conducted to do this. The time period was from 12:00 to 18:00 on a certain day, and the video data of the security camera (one) was analyzed by the motorcycle detection device 10 (the dynamic detection analysis program 50). A bicycle is detected from the moving image file (the image of the security camera) by the motorcycle detection device 10, and the detection information (moving image frame, screen position, etc.) is stored in the bicycle detection data file.

The following shows the result of summarizing the traffic status of the bicycle for each time zone from the bicycle detection data file. The total number of bicycles from 12:00 to 17:00 was 1,198, 199.7 per hour, and 3.3 per minute. The counting accuracy of the number of passing bicycles is 84.7% (1198 ÷ 1414 × 100) (count accuracy excluding 17:00 is 94.6%), and accuracy of general recognition rate by deep learning (See, for example, FIGS. 8 and 9).

  Regarding the difference between the number detected by the two-wheeled vehicle detection device 10 of the present invention (the dynamic detection analysis program 50) and the number detected visually, the two-wheeled vehicle detection device 10 of the present invention recognizes a bicycle that moves only in the upper (far) range of the screen. It is considered that the characteristic of the bicycle obtained from the video data is reduced and the recognition is no longer possible due to the inability to perform the operation (for example, see FIG. 9A) and the dark area at 17:00.

In addition, a demonstration experiment was performed using the motorcycle detection device 10 of the present invention to calculate the running speed of a bicycle detected from video data in a certain time zone. The running speed of the bicycle and the speed in each direction are converted into information such as <type>, <traveling direction>, <passing speed>, <riding / disembarking> from the “traffic data” created by the motorcycle detection device 10. It can be calculated based on As a result, the following table shows the number of bicycles passing at a speed of 10 km / h or more and their ratios.
The ratio of the number of bicycles passing at a speed of 10 km / h or more out of all the vehicles was 34/224 = 15.2%. As described above, since a bicycle running at a relatively high speed can be detected, the motorcycle detection device 10 of the present invention automatically detects a traveling bicycle in a pedestrian paradise such as a shopping street to determine danger. can do. Although the number of pedestrians (people) is not counted in this demonstration experiment, it is also possible to determine the danger based on the relationship between the number of pedestrians and the passing speed of the bicycle.

The calculation results of the bicycle speed for each traveling direction are shown in the following table.
By calculating the running speed of the bicycle for each direction, it can be seen that the average speed traveling east on a downhill is faster than the average speed traveling west on an uphill. Thus, the motorcycle detection device 10 of the present invention can calculate the traveling speed of the motorcycle including the bicycle for each direction.

  As described above, by using the motorcycle detection device and the like according to the present invention, it is possible to accurately detect a two-wheeled vehicle including a person, a motorcycle and a bicycle by detecting a dynamic appearing in a small image that cannot be detected by the object detection model based on the conventional deep learning. Can be detected. In other words, a camera such as a security camera installed at a shopping street or an intersection is an image where the shooting location is subtracted, that is, even when shooting from a bird's-eye view, whether the moving object in the image is a pedestrian, a bicycle or a motorcycle It can be accurately recognized as a two-wheeled vehicle. This makes it possible to automatically calculate the traffic amount and the traffic speed of the motorcycle at the shopping street or the intersection for each direction per unit time (for example, every hour).

  The motorcycle detection device and the like of the present invention can be used to automatically detect a person such as a pedestrian, a motorcycle such as a motorcycle or a bicycle from video data captured by a camera, and perform analysis or the like based on the detection result. It is.

Reference Signs List 10 motorcycle detection device 20 camera 30 storage device 40 storage device 50 dynamic detection analysis program 51 dynamic detection unit 52 image processing library 53 motorcycle detection unit 54 object detection algorithm 55 deep learning framework

Claims (14)

A motion detection unit that extracts a detection area that detects a motion from video data including a moving image or a continuous still image captured by a camera, and data corresponding to the detection area are input to an object detection model, and an object in the detection area is input. And a motorcycle detecting unit for detecting
The dynamic state detection unit performs image processing of enlargement or reduction in order to reduce data corresponding to the detection area to a preset size,
The two-wheeled vehicle detection device, wherein the object detected by the two-wheeled vehicle detection unit includes at least a two-wheeled vehicle including a bicycle and a motorcycle. The motorcycle detection device according to claim 1, wherein an identifier is assigned to the detection area when a motorcycle including at least a bicycle and a motorcycle is detected from the detection area by the object detection model. The identifier, the detection area extracted this time, before the detection area is extracted, comparing the previous detection area was extracted, within the range of a preset individual identification time, In the case where the distance is within the range of the preset individual identification distance, the identifier is the same as the previous detection area, and otherwise, the identifier is a new identifier. The motorcycle detection device according to any one of the preceding claims. The detection area is a difference area data within a predetermined detection maximum area and a range equal to or more than a predetermined detection minimum area,
4. The difference area data according to claim 1, wherein the difference area data is data having an area inside the contour of the difference data obtained by extracting a difference between a frame of the video data and a subsequent frame. Motorcycle detection device. The two-wheeled vehicle detection device according to claim 4, wherein the difference data is processed into a smooth image by image processing of expansion or contraction. The two-wheeled vehicle detection device according to any one of claims 1 to 5, wherein the detection area is subjected to image analysis and adjustment of brightness and contrast is performed. The detection area is stored in the detection database as a record in which, in addition to the ID corresponding to the identifier, a detection time, a type indicating a bicycle or a motorcycle, and coordinates of a center point of the detection area are recorded. The motorcycle detection device according to 2 or 3. In the detection database, a traveling direction of a bicycle or a motorcycle represented by the type is determined based on a locus of movement of the center point coordinates from an old record at the detection time to a new record at the detection time for each ID. The two-wheeled vehicle detection device according to claim 7, wherein: In the detection database, from a plurality of records specified by the ID, a center point coordinate in a record recorded first in time series is set as a start point, and a center point coordinate in a record recorded last is set as an end point. And the end point, perform a projective transformation based on a preset projective transformation coordinate,
Calculate an estimated distance based on the distance between pixels set in advance from the coordinates of the start point and the end point after the projective transformation, and the detection time is recorded in the record including the start point and the record including the end point. From the detected time, the travel time is calculated, and the passing speed of the bicycle or motorcycle represented by the type is estimated,
The two-wheeled vehicle detection device according to claim 8, wherein the preset projective transformation coordinates are coordinates obtained by projecting a scene captured by the camera as viewed from directly above. By comparing the passing speed of the bicycle or motorcycle represented by the type with a preset boarding / disembarking determination speed, to determine the boarding / disembarking state,
If the passing speed is equal to or higher than a preset boarding / disembarking determination speed, the boarding / disembarking state indicates that the vehicle is boarding,
The two-wheeled vehicle detection device according to claim 9, wherein if the speed is lower than the preset boarding / disembarking determination speed, the boarding / disembarking state indicates that the vehicle is getting off. Based on the traveling direction, the passing speed, and the boarding / disembarking state, at least one value of the number of traffic per unit time per direction, the boarding / dismounting ratio by direction, and the traveling speed by direction is calculated. The motorcycle detection device according to claim 10. The two-wheeled vehicle detection device according to any one of claims 1 to 11, wherein the object detection model is a convolutional neural network (CNN).   A two-wheeled vehicle detection program characterized by causing a computer to function as the two-wheeled vehicle detection device according to any one of claims 1 to 12 when executed by the computer.   A motorcycle comprising: a storage device for acquiring and storing moving image or continuous still image data from a camera; and a computer connected to the storage device via a network and executing the program according to claim 13. Detection system.