CN109800696A - Monitoring method, system and the computer readable storage medium of target vehicle - Google Patents
Monitoring method, system and the computer readable storage medium of target vehicle Download PDFInfo
- Publication number
- CN109800696A CN109800696A CN201910020540.9A CN201910020540A CN109800696A CN 109800696 A CN109800696 A CN 109800696A CN 201910020540 A CN201910020540 A CN 201910020540A CN 109800696 A CN109800696 A CN 109800696A
- Authority
- CN
- China
- Prior art keywords
- target
- target vehicle
- band
- vehicle
- detection model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 101
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000001514 detection method Methods 0.000 claims abstract description 231
- 238000012549 training Methods 0.000 claims description 31
- 230000001629 suppression Effects 0.000 claims description 25
- 238000013527 convolutional neural network Methods 0.000 claims description 19
- 230000006870 function Effects 0.000 claims description 18
- 238000004422 calculation algorithm Methods 0.000 claims description 14
- 230000004048 modification Effects 0.000 claims description 14
- 238000012986 modification Methods 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 9
- 238000012545 processing Methods 0.000 claims 1
- 238000011897 real-time detection Methods 0.000 abstract 1
- 238000003475 lamination Methods 0.000 description 11
- 238000004364 calculation method Methods 0.000 description 7
- 238000004140 cleaning Methods 0.000 description 6
- 230000007613 environmental effect Effects 0.000 description 6
- 238000007689 inspection Methods 0.000 description 6
- 230000000903 blocking effect Effects 0.000 description 4
- 239000003245 coal Substances 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000003912 environmental pollution Methods 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 235000013399 edible fruits Nutrition 0.000 description 2
- 239000003344 environmental pollutant Substances 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 231100000719 pollutant Toxicity 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000005406 washing Methods 0.000 description 2
- 230000033228 biological regulation Effects 0.000 description 1
- 239000003818 cinder Substances 0.000 description 1
- 239000002817 coal dust Substances 0.000 description 1
- 239000004035 construction material Substances 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- -1 dregs Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Landscapes
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
The present invention provides a kind of monitoring method of target vehicle, system and computer readable storage mediums；Whether the monitoring method of target vehicle includes: in each frame image by video input to be detected to target vehicle detection model, to confirm video to be detected comprising target vehicle；When in video to be detected including target vehicle, the band of position of target vehicle is obtained by the detection block of target vehicle detection model；Tracing positional region；Judge whether the band of position is within the scope of target area；When the band of position is within the scope of target area, the stay time passed through when the band of position is within the scope of target area is obtained；When stay time is less than duration threshold value, target vehicle is labeled as irregularity vehicle.It applies technical solution provided by the invention realizes automatic, the real-time detection for whether meeting dump truck monitoring of environment requirement, ensure that the accuracy of the detection and tracking of dump truck.
Description
Technical field
The present invention relates to vehicle monitoring technical fields, monitoring method, a kind of mesh in particular to a kind of target vehicle
The monitoring system of mark vehicle and a kind of computer readable storage medium.
Background technique
Currently, in the related art, video monitoring is always the necessary means of urban transportation and monitoring of environment.Wherein,
Dump truck is different from daily manned car or car and is usually used in loading, and the regions such as construction site, industrial and mining enterprises are made
The dump truck of industry usually carry construction material, dregs, coal, ore, dregs, chemical substance etc.；These dump trucks are monitoring of environment
Emphasis.However it is different from general traffic video monitoring to the monitoring of dump truck.It is the traffic video in monitoring demand first
Whether the vehicle that monitoring is mainly used for detecting on urban road violates the traffic regulations, and is mainly used for the monitoring of dump truck
Whether detection dump truck meets the requirement of environmental protection in transport cargo.It is generally all wanted near the entrance of coal mine building site vehicle
It asks and must have available service rack, it is desirable that the loading dump truck of discrepancy must when passing through from the service rack for being covered with water injector
It must stay for some time to guarantee that cinder on dump truck vehicle body and coal dust by again clean, are avoided bringing dirt to urban road
Dye.And how to realize to detect by detection model and confirm that dump truck has abided by this requirement and stopped time enough in service rack
It is a technical problem to be solved urgently.
Environmentally friendly associated monitoring demand is detected and realized for dump truck based on video, detection model is promoted and detects mud
Head vehicle accuracy rate and count dump truck the residence time of service rack whether meet environmental requirement be solve above-mentioned technical problem
Main direction of studying.In currently used target detection model, non-maximum suppression (Non-maximum suppression,
NMS) object detection model is important component part, it is based on object detection score and generates detection block, the highest detection of score
The detection block of target is selected display, and other detection blocks for having overlapping are suppressed.But since dump truck is being taken the photograph in many cases
It is mutually blocked as existing in head, carries out the feelings that post-processing causes the dump truck being blocked to be missed using non-maximum suppression at this time
Condition reduces the detection accuracy of detection model.Meanwhile general algorithm of target detection can not be detected apart from farther away dump truck
Target brings difficulty to the accuracy of the detection and tracking of dump truck, the mistake for causing the time to count.
Summary of the invention
The present invention is directed to solve at least one of the technical problems existing in the prior art or related technologies.
For this purpose, the first aspect of the present invention proposes a kind of monitoring method of target vehicle.
The second aspect of the present invention proposes a kind of monitoring system of target vehicle.
The third aspect of the present invention proposes a kind of computer readable storage medium.
In view of this, the first aspect of the present invention provides a kind of monitoring method of target vehicle, comprising: by view to be detected
Whether frequency is input to target vehicle detection model, include target vehicle in each frame image to confirm video to be detected；When to
When detecting in video comprising target vehicle, the detection block for passing through target vehicle detection model obtains the band of position of target vehicle；
Tracing positional region；Judge whether the band of position is within the scope of target area；When the band of position is within the scope of target area
When, obtain the stay time passed through when the band of position is within the scope of target area；It, will when stay time is less than duration threshold value
Target vehicle is labeled as irregularity vehicle.
In the technical scheme, it obtains video to be detected and is input to target vehicle detection model, first determine whether to be detected
Whether there is target vehicle in video, after target vehicle occurs, is marked and obtained locating for target vehicle by detection block
The band of position, the band of position of the real-time tracking target vehicle, until the band of position is completely within the scope of target area, i.e. mesh
When mark vehicle is located within service rack, timing is opened to obtain target vehicle and is located at the stay time passed through in service rack, such as
Fruit stay time is less than duration threshold value, then shows the dirt that target vehicle cleans the sufficiently long time without service rack, on vehicle body
Dye object is unable to get effective cleaning, therefore the target vehicle is labeled as irregularity vehicle, and turn in a report to monitoring system.
Technical solution provided by the invention is applied, the dump truck in monitor video is accurately identified by target vehicle detection model,
And the position of dump truck is tracked, judge whether the position of dump truck is in the position range of service rack, and be in dump truck
When within the scope of service rack record dump truck residence time, realize to dump truck whether meet monitoring of environment requirement it is automatic,
Detection in real time, ensure that the accuracy of the detection and tracking of dump truck, realizes the intelligence of urban transportation and monitoring of environment
Change.
Specifically, the monitor video for obtaining the entrance of coal mine building site vehicle in real time, inputs target for monitor video in real time
Vehicle detection model, target vehicle detection model first determine whether dump truck occur in current monitor video, work as target carriage
When detection model recognizes dump truck, the characteristic area of dump truck is marked by detection block, and obtains the dump truck in video
Position area information in image；By the position area information of algorithm real-time tracking dump truck, the position area of dump truck is judged
Whether domain is located within the scope of the band of position of the service rack marked in advance, when the band of position of dump truck is located at the position of service rack
When in regional scope, the stay time that timing dump truck is passed through in service rack when opening, when the band of position of dump truck is left
When the band of position range of service rack, stop timing, using the period between time zero and timing end point as dump truck
The cleaning duration of stay time, i.e. dump truck in service rack.Judge whether stay time is less than duration threshold value, that is, judges mud head
Whether vehicle has cleaned the sufficiently long time to guarantee that the pollutant on dump truck vehicle body has been cleaned, if stopping in service rack
It stays duration to be less than duration threshold value, then illustrates that the dump truck does not have meet the requirement of environmental protection, mark the dump truck at this time and obtain the mud
The information such as license plate number, vehicle, the vehicle color of head vehicle are simultaneously sent to monitor supervision platform realization to the label note of irregularity target vehicle
Record.
In addition, the monitoring method of the target vehicle in above-mentioned technical proposal provided by the invention can also have following add
Technical characteristic:
In the above-mentioned technical solutions, it is preferable that the step of by video input to be detected to target vehicle detection model it
Before further include: modification target detection model is to obtain modified target detection model；Monitor video is acquired, monitor video is passed through
Obtain the frame image comprising target vehicle；It is labeled using characteristic area of the detection block to the target vehicle in frame image, with
Form training dataset；By the modified target detection model of training dataset training, to obtain target vehicle detection model.
In the technical scheme, general target detection model is obtained, it is preferable that select SSD (single shot
Multibox detection, a kind of algorithm of target detection) model modifies as target detection model, and to it, with
To modified detection model.A large amount of acquisition monitor video data, extracting includes target vehicle in monitor video, i.e. dump truck
Frame image is labeled the characteristic area of the dump truck of frame image by the detection block of modified detection model, specifically,
The headstock region of dump truck as characteristic area and can be labeled, pass through the frame image making training data marked
Collection, wherein the quantity of the frame image marked, which is maintained at 20000 or more, can be obtained preferable effect, pass through the training made
The modified detection model of data set training, to obtain target vehicle detection model.It is trained according to the technical solution of the present invention
The target detection model arrived has the characteristics that recognition accuracy is high, can be effectively reduced general to the missing inspection of the dump truck mutually blocked
Rate.
In any of the above-described technical solution, it is preferable that modification target detection model is to obtain modified target detection mould
The step of type, specifically includes: choosing target detection model, increases at least two in the convolutional neural networks end of target detection model
The symmetrical warp lamination of layer；The non-maxima suppression function formula of target detection model is modified are as follows:
Wherein, siFor the confidence level of detection block, iou (M, bi) be detection block friendship and ratio, NpAnd NqFor for determining iou
(M,bi) value range constant.
In the technical scheme, in the convolutional neural networks end of the general target detection model of selection, i.e. CNN
(Convolutional Neural Networks, convolutional neural networks) at least two layers symmetrical deconvolution of network end-point increase
Layer, it is preferable that increase by 3 layers of symmetrical warp lamination, and modify former non-maxima suppression function NMS (Non-maximum
Suppression, non-maximum suppression)
Are as follows:
Wherein, siFor the confidence level of detection block, iou (M, bi) be detection block friendship and ratio, NpAnd NqFor for determining iou
(M,bi) value range constant, compared to former non-maxima suppression function, modified non-maxima suppression function is increased
Threshold value, and reset according to the threshold value and reduce confidence level as original confidence levelRather than it is set to 0.It is repaired by above-mentioned
Changing can be effectively avoided is inhibited directly apart from target vehicle that is very close or mutually blocking between each other, and then improves and know
Other accuracy rate reduces the false dismissal probability to the dump truck mutually blocked.
In any of the above-described technical solution, it is preferable that the step of tracing positional region, specifically: use KCF (Kernel
Correlation Filter, core correlation filtering) algorithm keeps track target vehicle, to obtain the real-time position area of target vehicle
Domain.
In the technical scheme, using the real time position of KCF algorithm keeps track target vehicle, calculation amount is small, and tracking result is quasi-
Really, can effectively improve and dump truck position is accurately tracked, so ensure that dump truck the band of position detect and with
The accuracy of track.
In any of the above-described technical solution, it is preferable that in the step for judging whether the band of position is within the scope of target area
Before rapid, further includes: the label target regional scope in video to be detected, to obtain the coordinate information of target area range；With
And judge the step whether band of position is within the scope of target area, it specifically includes: obtaining the center point coordinate of the band of position；
When center point coordinate meets preset formula, determine that the band of position is within the scope of target area.
In the technical scheme, after getting video to be detected, the position area of service rack is marked out in detection video
Domain as target area range, and obtains the coordinate information in target zone region.In the band of position for getting target vehicle
Afterwards, the center point coordinate in calculating position region is sentenced by the coordinate information of preset formula, center point coordinate and preset range region
Whether the disconnected band of position is within the scope of target area, i.e. whether dump truck enters service rack, so realize to dump truck whether
Stay long enough is judged in service rack, and marks irregularity vehicle of the stay time less than duration threshold value.
In any of the above-described technical solution, it is preferable that be obtained by the following formula the center point coordinate of target vehicle:
Wherein, coordinate, (x are put centered on (x, y)p,yp) be the band of position upper left corner coordinate, w be the band of position width
Degree, h are the height of the band of position.
In the technical scheme, using the current frame image of target vehicle detection model inspection video to be detected, current
There are the coordinate informations for when dump truck, exporting all dump truck headstock positions in picture detected in frame image.Specifically,
Assuming that M platform dump truck is detected altogether, wherein the coordinate of the corresponding detection block of i-th dump truck is ci(xi,yi,wi,hi)；Wherein
(xi,yi) respectively indicate detection block the point of top left co-ordinate in the picture position, w and h respectively indicate detection block in the picture
Width and height.By tracking box track i-th dump truck detection block coordinate, and by above-mentioned formula calculating detection block or
The center point coordinate (x, y) of person's tracking box, when the center point coordinate (x, y) of detection block or tracking box meets preset formula,
Determine that the band of position is within the scope of target area, i.e., whether dump truck enters service rack.
In any of the above-described technical solution, it is preferable that preset formula specifically:
Wherein, coordinate, (x are put centered on (x, y)min,ymin) be target area upper left angle point coordinate, wdFor target area
Width, hdFor the height of target area.
In the technical scheme, the coordinate (x of the upper left angle point of the target area range of mark is obtainedmin,ymin), go forward side by side one
Step obtains the width w of target areadWith the height h of target aread, when center point coordinate (x, y) meets above-mentioned formula, determine
The band of position is within the scope of target area, i.e., whether dump truck enters service rack.
In any of the above-described technical solution, it is preferable that the acquisition band of position was passed through when being within the scope of target area stops
The step of staying duration specifically includes: when center point coordinate starts to meet preset formula, record current point in time is at the first time
Point；At interval of default frame image, repetition judges whether center point coordinate meets preset formula；When center point coordinate starts to be unsatisfactory for
When preset formula, record current point in time was the second time point；Stay time is calculated by first time point and the second time point.
In the technical scheme, when center point coordinate starts to meet preset formula, illustrate that target vehicle enters target
Regional scope, i.e. dump truck enter service rack, and record current point in time is first time point lmin, at interval of default frame image,
Repetition judges whether center point coordinate meets preset formula, it is preferable that repeats judge whether center point coordinate is full at interval of 30 frames
Sufficient preset formula is primary, in the case where guaranteeing Detection accuracy reduce calculate pressure, when center point coordinate start to be unsatisfactory for it is pre-
If when formula, illustrating that target vehicle has left target area range, i.e. dump truck has left service rack, and record current point in time is made
For the second time point lmax, the stay time of target vehicle can be calculated by first time point and the second time point, and then judge
Whether the dump truck is irregularity vehicle.
In any of the above-described technical solution, it is preferable that calculate stay time by first time point and the second time point
Formula, specifically:
T=lmax-lmin；
Wherein, T is stay time, lmaxFor the second time point, lminFor first time point.
In the technical scheme, target vehicle leaves the time point of target area range and target vehicle enters target area
Elapsed time section between the time point of range, i.e. stay time of the dump truck in service rack.
In any of the above-described technical solution, it is preferable that the monitoring method of target vehicle further include: statistics irregularity vehicle
Quantity；When the quantity of irregularity vehicle is more than the accounting of quantity monitoring threshold value and/or irregularity vehicle in target complete vehicle
When monitoring threshold value more than ratio, issue warning signal.
In the technical scheme, the quantity of the dump truck of all irregularities is counted, if the quantity of the dump truck of irregularity is super
When crossing accounting of the dump truck in whole dump trucks of quantity monitoring threshold value and/or irregularity and being more than ratio monitoring threshold value, to prison
Control system issues warning signal, and a large amount of irregularity vehicles occurs in warning, to realize the prompt of environmental pollution risk.
Second aspect of the present invention provides a kind of monitoring system of target vehicle, including memory and processor；Memory
For storing computer program, processor for execute the computer program with: by video input to be detected to target vehicle
Whether detection model includes target vehicle in each frame image to confirm video to be detected；When in video to be detected include mesh
When marking vehicle, the band of position of target vehicle is obtained by the detection block of target vehicle detection model；Tracing positional region；Judgement
Whether the band of position is within the scope of target area；When the band of position is within the scope of target area, obtain at the band of position
The stay time passed through when within the scope of target area；When stay time is less than duration threshold value, by target vehicle labeled as not
Close rule vehicle.
In the technical scheme, it obtains video to be detected and is input to target vehicle detection model, first determine whether to be detected
Whether there is target vehicle in video, after target vehicle occurs, is marked and obtained locating for target vehicle by detection block
The band of position, the band of position of the real-time tracking target vehicle, until the band of position is completely within the scope of target area, i.e. mesh
When mark vehicle is located within service rack, timing is opened to obtain target vehicle and is located at the stay time passed through in service rack, such as
Fruit stay time is less than duration threshold value, then shows the dirt that target vehicle cleans the sufficiently long time without service rack, on vehicle body
Dye object is unable to get effective cleaning, therefore the target vehicle is labeled as irregularity vehicle, and turn in a report to monitoring system.
Technical solution provided by the invention is applied, the dump truck in monitor video is accurately identified by target vehicle detection model,
And the position of dump truck is tracked, judge whether the position of dump truck is in the position range of service rack, and be in dump truck
When within the scope of service rack record dump truck residence time, realize to dump truck whether meet monitoring of environment requirement it is automatic,
Detection in real time, ensure that the accuracy of the detection and tracking of dump truck, realizes the intelligence of urban transportation and monitoring of environment
Change.
In any of the above-described technical solution, it is preferable that processor is also used to: after modification target detection model is to obtain modification
Target detection model；Monitor video is acquired, the frame image comprising target vehicle is obtained by monitor video；Use detection block pair
The characteristic area of target vehicle in frame image is labeled, to form training dataset；It is modified by training dataset training
Target detection model afterwards, to obtain target vehicle detection model.
In the technical scheme, general target detection model is obtained, it is preferable that select SSD (single shot
Multibox detection) model modifies as target detection model, and to it, to obtain modified detection mould
Type.A large amount of acquisition monitor video data, extracting in monitor video includes target vehicle, i.e. the frame image of dump truck, passes through modification
The detection block of detection model afterwards is labeled the characteristic area of the dump truck of frame image, specifically, can be by dump truck
Headstock region is as characteristic area and is labeled, and passes through the frame image making training dataset marked, wherein marked
The quantity of frame image, which is maintained at 20000 or more, can be obtained preferable effect, after being modified by the training dataset training made
Detection model, to obtain target vehicle detection model.The target detection model that training obtains according to the technical solution of the present invention
Have the characteristics that recognition accuracy is high, the false dismissal probability to the dump truck mutually blocked can be effectively reduced.
In any of the above-described technical solution, it is preferable that processor is also used to: target detection model is chosen, in target detection
The convolutional neural networks end of model increases at least two layers symmetrical warp lamination；By the non-maxima suppression of target detection model
Function formula modification are as follows:
Wherein, siFor the confidence level of detection block, iou (M, bi) be detection block friendship and ratio, NpAnd NqFor for determining iou
(M,bi) value range constant.
In the technical scheme, in the convolutional neural networks end of the general target detection model of selection, i.e. CNN
(Convolutional Neural Networks) at least two layers of network end-point increase symmetrical warp lamination, it is preferable that increase
3 layers of symmetrical warp lamination, and by former non-maxima suppression function NMS (Non-maximum suppression):
Modification are as follows:
Wherein, siFor the confidence level of detection block, iou (M, bi) be detection block friendship and ratio, NpAnd NqFor for determining iou
(M,bi) value range constant, compared to former non-maxima suppression function, modified non-maxima suppression function is increased
Threshold value, and reset according to the threshold value and reduce confidence level as original confidence levelRather than it is set to 0.It is repaired by above-mentioned
Changing can be effectively avoided is inhibited directly apart from target vehicle that is very close or mutually blocking between each other, and then improves and know
Other accuracy rate reduces the false dismissal probability to the dump truck mutually blocked.
In any of the above-described technical solution, it is preferable that processor is also used to: KCF algorithm keeps track target vehicle is used, with
To the real-time band of position of target vehicle.
In the technical scheme, using the real time position of KCF algorithm keeps track target vehicle, calculation amount is small, and tracking result is quasi-
Really, can effectively improve and dump truck position is accurately tracked, so ensure that dump truck the band of position detect and with
The accuracy of track.
In any of the above-described technical solution, it is preferable that processor is also used to: the label target region model in video to be detected
It encloses, to obtain the coordinate information of target area range；Obtain the center point coordinate of the band of position；It is preset when center point coordinate meets
When formula, determine that the band of position is within the scope of target area.
In the technical scheme, after getting video to be detected, the position area of service rack is marked out in detection video
Domain as target area range, and obtains the coordinate information in target zone region.In the band of position for getting target vehicle
Afterwards, the center point coordinate in calculating position region is sentenced by the coordinate information of preset formula, center point coordinate and preset range region
Whether the disconnected band of position is within the scope of target area, i.e. whether dump truck enters service rack, so realize to dump truck whether
Stay long enough is judged in service rack, and marks irregularity vehicle of the stay time less than duration threshold value.
In any of the above-described technical solution, it is preferable that be obtained by the following formula the center point coordinate of target vehicle:
Wherein, coordinate, (x are put centered on (x, y)p, yp) be the band of position upper left corner coordinate, w be the band of position width
Degree, h are the height of the band of position.
In the technical scheme, using the current frame image of target vehicle detection model inspection video to be detected, current
There are the coordinate informations for when dump truck, exporting all dump truck headstock positions in picture detected in frame image.Specifically,
Assuming that M platform dump truck is detected altogether, wherein the coordinate of the corresponding detection block of i-th dump truck is ci(xi,yi,wi,hi)；Wherein
(xi,yi) respectively indicate detection block the point of top left co-ordinate in the picture position, w and h respectively indicate detection block in the picture
Width and height.By tracking box track i-th dump truck detection block coordinate, and by above-mentioned formula calculating detection block or
The center point coordinate (x, y) of person's tracking box, when the center point coordinate (x, y) of detection block or tracking box meets preset formula,
Determine that the band of position is within the scope of target area, i.e., whether dump truck enters service rack.
In any of the above-described technical solution, it is preferable that preset formula specifically:
Wherein, coordinate, (x are put centered on (x, y)min,ymin) be target area upper left angle point coordinate, wdFor target area
Width, hdFor the height of target area.
In the technical scheme, the coordinate (x of the upper left angle point of the target area range of mark is obtainedmin,ymin), go forward side by side one
Step obtains the width w of target areadWith the height h of target aread, when center point coordinate (x, y) meets above-mentioned formula, determine
The band of position is within the scope of target area, i.e., whether dump truck enters service rack.
In any of the above-described technical solution, it is preferable that processor is also used to: when center point coordinate starts to meet preset formula
When, record current point in time is first time point；At interval of default frame image, it is default that repetition judges whether center point coordinate meets
Formula；When center point coordinate starts to be unsatisfactory for preset formula, record current point in time was the second time point；Pass through at the first time
Point and the second time point calculate stay time.
In the technical scheme, when center point coordinate starts to meet preset formula, illustrate that target vehicle enters target
Regional scope, i.e. dump truck enter service rack, and record current point in time is first time point lmin, at interval of default frame image,
Repetition judges whether center point coordinate meets preset formula, it is preferable that repeats judge whether center point coordinate is full at interval of 30 frames
Sufficient preset formula is primary, in the case where guaranteeing Detection accuracy reduce calculate pressure, when center point coordinate start to be unsatisfactory for it is pre-
If when formula, illustrating that target vehicle has left target area range, i.e. dump truck has left service rack, and record current point in time is made
For the second time point lmax, the stay time of target vehicle can be calculated by first time point and the second time point, and then judge
Whether the dump truck is irregularity vehicle.
In any of the above-described technical solution, it is preferable that calculate stay time by first time point and the second time point
Formula, specifically:
T=lmax-lmin；
Wherein, T is stay time, lmaxFor the second time point, lminFor first time point.
In the technical scheme, target vehicle leaves the time point of target area range and target vehicle enters target area
Elapsed time section between the time point of range, i.e. stay time of the dump truck in service rack.
In any of the above-described technical solution, it is preferable that processor is also used to: the quantity of statistics irregularity vehicle；When not conforming to
The quantity for advising vehicle is more than that the accounting of quantity monitoring threshold value and/or irregularity vehicle in target complete vehicle is supervised more than ratio
When controlling threshold value, issue warning signal.
In the technical scheme, the quantity of the dump truck of all irregularities is counted, if the quantity of the dump truck of irregularity is super
When crossing accounting of the dump truck in whole dump trucks of quantity monitoring threshold value and/or irregularity and being more than ratio monitoring threshold value, to prison
Control system issues warning signal, and a large amount of irregularity vehicles occurs in warning, to realize the prompt of environmental pollution risk.
Third aspect present invention provides a kind of computer readable storage medium, is stored thereon with computer program, calculates
The monitoring method of the target vehicle as described in any of the above-described technical solution, therefore the meter are realized when machine program is executed by processor
Calculation machine readable storage medium storing program for executing includes whole beneficial effects of the monitoring method of target vehicle described in any of the above-described technical solution.
Additional aspect and advantage of the invention will be set forth in part in the description, and will partially become from the following description
Obviously, or practice through the invention is recognized.
Detailed description of the invention
Above-mentioned and/or additional aspect of the invention and advantage will become from the description of the embodiment in conjunction with the following figures
Obviously and it is readily appreciated that, in which:
Fig. 1 shows the flow chart of the monitoring method of target vehicle according to an embodiment of the invention；
Fig. 2 shows the flow charts of the monitoring method of target vehicle according to another embodiment of the invention；
Fig. 3 shows the flow chart of the monitoring method of target vehicle according to still another embodiment of the invention；
Fig. 4 shows the flow chart of the monitoring method of the target vehicle of still another embodiment in accordance with the present invention；
Fig. 5 shows the flow chart of the monitoring method of the target vehicle of still another embodiment in accordance with the present invention；
Fig. 6 shows the flow chart of the monitoring method of the target vehicle of still another embodiment in accordance with the present invention；
Fig. 7 shows the block diagram of the monitoring system of target vehicle according to an embodiment of the invention.
Specific embodiment
To better understand the objects, features and advantages of the present invention, with reference to the accompanying drawing and specific real
Applying mode, the present invention is further described in detail.It should be noted that in the absence of conflict, the implementation of the application
Feature in example and embodiment can be combined with each other.
In the following description, numerous specific details are set forth in order to facilitate a full understanding of the present invention, still, the present invention may be used also
To be implemented using other than the one described here other modes, therefore, protection scope of the present invention is not by described below
Specific embodiment limitation.
Monitoring method, the target carriage of the target vehicle described according to some embodiments of the invention are described referring to Fig. 1 to Fig. 7
Monitoring system and computer readable storage medium.
As shown in Figure 1, providing a kind of monitoring method of target vehicle in the embodiment of first aspect present invention, wrap
It includes:
S102, by video input to be detected to target vehicle detection model, to confirm each frame image of video to be detected
In whether include target vehicle；
S104 obtains mesh by the detection block of target vehicle detection model when in video to be detected including target vehicle
Mark the band of position of vehicle；
S106, tracing positional region；
S108, judges whether the band of position is within the scope of target area；
S110, when the band of position is within the scope of target area, when the acquisition band of position is within the scope of target area
The stay time of process；
Target vehicle is labeled as irregularity vehicle when stay time is less than duration threshold value by S112.
In this embodiment, it obtains video to be detected and is input to target vehicle detection model, first determine whether view to be detected
Whether there is target vehicle in frequency, after target vehicle occurs, is marked by detection block and obtain position locating for target vehicle
Set region, the band of position of the real-time tracking target vehicle, until the band of position is completely within the scope of target area, i.e. target
When vehicle is located within service rack, timing is opened to obtain target vehicle and is located at the stay time passed through in service rack, if
Stay time is less than duration threshold value, then shows the pollution that target vehicle cleans the sufficiently long time without service rack, on vehicle body
Object is unable to get effective cleaning, therefore the target vehicle is labeled as irregularity vehicle, and turn in a report to monitoring system.It answers
With technical solution provided by the invention, the dump truck in monitor video is accurately identified by target vehicle detection model, and
The position for tracking dump truck, judges whether the position of dump truck is in the position range of service rack, and is in and washes in dump truck
The residence time that dump truck is recorded when within the scope of ride realizes to whether dump truck meets the automatic, real of monitoring of environment requirement
When detection, ensure that the accuracy of the detection and tracking of dump truck, realize the intelligence of urban transportation and monitoring of environment.
Specifically, the monitor video for obtaining the entrance of coal mine building site vehicle in real time, inputs target for monitor video in real time
Vehicle detection model, target vehicle detection model first determine whether dump truck occur in current monitor video, work as target carriage
When detection model recognizes dump truck, the characteristic area of dump truck is marked by detection block, and obtains the dump truck in video
Position area information in image；By the position area information of algorithm real-time tracking dump truck, the position area of dump truck is judged
Whether domain is located within the scope of the band of position of the service rack marked in advance, when the band of position of dump truck is located at the position of service rack
When in regional scope, the stay time that timing dump truck is passed through in service rack when opening, when the band of position of dump truck is left
When the band of position range of service rack, stop timing, using the period between time zero and timing end point as dump truck
The cleaning duration of stay time, i.e. dump truck in service rack.Judge whether stay time is less than duration threshold value, that is, judges mud head
Whether vehicle has cleaned the sufficiently long time to guarantee that the pollutant on dump truck vehicle body has been cleaned, if stopping in service rack
It stays duration to be less than duration threshold value, then illustrates that the dump truck does not have meet the requirement of environmental protection, mark the dump truck at this time and obtain the mud
The information such as license plate number, vehicle, the vehicle color of head vehicle are simultaneously sent to monitor supervision platform realization to the label note of irregularity target vehicle
Record.
In one embodiment of the invention, it is preferable that as shown in Fig. 2, the monitoring method of target vehicle, comprising:
S202 modifies target detection model to obtain modified target detection model；
S204 acquires monitor video, obtains the frame image comprising target vehicle by monitor video；
S206 is labeled, to form training data using characteristic area of the detection block to the target vehicle in frame image
Collection；
S208, by the modified target detection model of training dataset training, to obtain target vehicle detection model；
S210, by video input to be detected to target vehicle detection model, to confirm each frame image of video to be detected
In whether include target vehicle；
S212 obtains mesh by the detection block of target vehicle detection model when in video to be detected including target vehicle
Mark the band of position of vehicle；
S214, tracing positional region；
S216, judges whether the band of position is within the scope of target area；
S218, when the band of position is within the scope of target area, when the acquisition band of position is within the scope of target area
The stay time of process；
Target vehicle is labeled as irregularity vehicle when stay time is less than duration threshold value by S220.
In this embodiment, general target detection model is obtained, it is preferable that select SSD (single shot
Multibox detection) model modifies as target detection model, and to it, to obtain modified detection mould
Type.A large amount of acquisition monitor video data, extracting in monitor video includes target vehicle, i.e. the frame image of dump truck, passes through modification
The detection block of detection model afterwards is labeled the characteristic area of the dump truck of frame image, specifically, can be by dump truck
Headstock region is as characteristic area and is labeled, and passes through the frame image making training dataset marked, wherein marked
The quantity of frame image, which is maintained at 20000 or more, can be obtained preferable effect, after being modified by the training dataset training made
Detection model, to obtain target vehicle detection model.The target detection model that training obtains according to the technical solution of the present invention
Have the characteristics that recognition accuracy is high, the false dismissal probability to the dump truck mutually blocked can be effectively reduced.
In one embodiment of the invention, it is preferable that modification target detection model is to obtain modified target detection
The step of model, specifically includes: choosing target detection model, increases at least in the convolutional neural networks end of target detection model
Two layers of symmetrical warp lamination；The non-maxima suppression function formula of target detection model is modified are as follows:
Wherein, siFor the confidence level of detection block, iou (M, bi) be detection block friendship and ratio, NpAnd NqFor for determining iou
(M,bi) value range constant.
In this embodiment, in the convolutional neural networks end of the general target detection model of selection, i.e. CNN
(Convolutional Neural Networks) at least two layers of network end-point increase symmetrical warp lamination, it is preferable that increase
3 layers of symmetrical warp lamination, and by former non-maxima suppression function NMS (Non-maximum suppression)
Modification are as follows:
Wherein, siFor the confidence level of detection block, iou (M, bi) be detection block friendship and ratio, NpAnd NqFor for determining iou
(M,bi) value range constant, compared to former non-maxima suppression function, modified non-maxima suppression function is increased
Threshold value, and reset according to the threshold value and reduce confidence level as original confidence levelRather than it is set to 0.It is repaired by above-mentioned
Changing can be effectively avoided is inhibited directly apart from target vehicle that is very close or mutually blocking between each other, and then improves and know
Other accuracy rate reduces the false dismissal probability to the dump truck mutually blocked.
In one embodiment of the invention, it is preferable that the step of tracing positional region, specifically: using KCF algorithm with
Track target vehicle, to obtain the real-time band of position of target vehicle.
In this embodiment, using the real time position of KCF algorithm keeps track target vehicle, calculation amount is small, and tracking result is accurate,
It can effectively improve dump truck position is accurately tracked, and then ensure that the band of position ground detection and tracking of dump truck
Accuracy.
In one embodiment of the invention, it is preferable that as shown in figure 3, the monitoring method of target vehicle, comprising:
S302, the label target regional scope in video to be detected, to obtain the coordinate information of target area range；
S304, by video input to be detected to target vehicle detection model, to confirm each frame image of video to be detected
In whether include target vehicle；
S306 obtains mesh by the detection block of target vehicle detection model when in video to be detected including target vehicle
Mark the band of position of vehicle；
S308, tracing positional region；
S310 obtains the center point coordinate of the band of position；
S312 determines that the band of position is within the scope of target area when center point coordinate meets preset formula；
S314 obtains the stay time passed through when the band of position is within the scope of target area；
Target vehicle is labeled as irregularity vehicle when stay time is less than duration threshold value by S316.
In this embodiment, after getting video to be detected, the band of position of service rack is marked out in detection video,
As target area range, and obtain the coordinate information in target zone region.After getting the band of position of target vehicle, meter
The center point coordinate for calculating the band of position, judges position by the coordinate information of preset formula, center point coordinate and preset range region
Set whether region is within the scope of target area, i.e., whether dump truck enters service rack, and then realizes and whether washing to dump truck
Stay long enough is judged in ride, and marks irregularity vehicle of the stay time less than duration threshold value.
In one embodiment of the invention, it is preferable that be obtained by the following formula the center point coordinate of target vehicle:
Wherein, coordinate, (x are put centered on (x, y)p,yp) be the band of position upper left corner coordinate, w be the band of position width
Degree, h are the height of the band of position.
In this embodiment, using the current frame image of target vehicle detection model inspection video to be detected, in present frame
There are the coordinate informations for when dump truck, exporting all dump truck headstock positions in picture detected in image.Specifically, false
If detecting M platform dump truck altogether, wherein the coordinate of the corresponding detection block of i-th dump truck is ci(xi,yi,wi,hi)；Wherein (xi,
yi) respectively indicate detection block the point of top left co-ordinate in the picture position, w and h respectively indicate the width of detection block in the picture
Degree and height.By tracking box track i-th dump truck detection block coordinate, and by above-mentioned formula calculating detection block or
The center point coordinate (x, y) of tracking box is sentenced when the center point coordinate (x, y) of detection block or tracking box meets preset formula
Location area is within the scope of target area, i.e., whether dump truck enters service rack.
In one embodiment of the invention, it is preferable that preset formula specifically:
Wherein, coordinate, (x are put centered on (x, y)min,ymin) be target area upper left angle point coordinate, wdFor target area
Width, hdFor the height of target area.
In this embodiment, the coordinate (x of the upper left angle point of the target area range of mark is obtainedmin,ymin), and further
Obtain the width w of target areadWith the height h of target aread, when center point coordinate (x, y) meets above-mentioned formula, decision bits
It sets region to be within the scope of target area, i.e., whether dump truck enters service rack.
In one embodiment of the invention, it is preferable that as shown in figure 4, the monitoring method of target vehicle, comprising:
S402, by video input to be detected to target vehicle detection model, to confirm each frame image of video to be detected
In whether include target vehicle；
S404 obtains mesh by the detection block of target vehicle detection model when in video to be detected including target vehicle
Mark the band of position of vehicle；
S406, tracing positional region；
S408, judges whether the band of position is within the scope of target area；
S410, when center point coordinate starts to meet preset formula, record current point in time is first time point；
S412, at interval of default frame image, repetition judges whether center point coordinate meets preset formula；
S414, when center point coordinate starts to be unsatisfactory for preset formula, record current point in time was the second time point；
S416 calculates stay time by first time point and the second time point；
Target vehicle is labeled as irregularity vehicle when stay time is less than duration threshold value by S418.
In this embodiment, when center point coordinate starts to meet preset formula, illustrate that target vehicle enters target area
Domain range, i.e. dump truck enter service rack, and record current point in time is first time point lmin, at interval of default frame image, weight
Judge whether center point coordinate meets preset formula again, it is preferable that repeat to judge whether center point coordinate meets at interval of 30 frames
Preset formula is primary, reduces in the case where guaranteeing Detection accuracy and calculates pressure, when center point coordinate starts to be unsatisfactory for presetting
When formula, illustrate that target vehicle has left target area range, i.e. dump truck has left service rack, records current point in time conduct
Second time point lmax, the stay time of target vehicle can be calculated by first time point and the second time point, and then judging should
Whether dump truck is irregularity vehicle.
In one embodiment of the invention, it is preferable that stay time is calculated by first time point and the second time point
Formula, specifically:
T=lmax-lmin；
Wherein, T is stay time, lmaxFor the second time point, lminFor first time point.
In this embodiment, target vehicle leaves the time point of target area range and target vehicle enters target area model
Elapsed time section between the time point enclosed, i.e. stay time of the dump truck in service rack, can be with by calculating stay time
Judge whether current dump truck is irregularity vehicle.
In one embodiment of the invention, it is preferable that as shown in figure 5, the monitoring method of target vehicle, comprising:
S502, by video input to be detected to target vehicle detection model, to confirm each frame image of video to be detected
In whether include target vehicle；
S504 obtains mesh by the detection block of target vehicle detection model when in video to be detected including target vehicle
Mark the band of position of vehicle；
S506, tracing positional region；
S508, judges whether the band of position is within the scope of target area；
S510, when the band of position is within the scope of target area, when the acquisition band of position is within the scope of target area
The stay time of process；
Target vehicle is labeled as irregularity vehicle when stay time is less than duration threshold value by S512；
S514 counts the quantity of irregularity vehicle；
S516, when the quantity of irregularity vehicle is more than quantity monitoring threshold value and/or irregularity vehicle in target complete vehicle
In accounting be more than ratio monitoring threshold value when, issue warning signal.
In this embodiment, the quantity of the dump truck of all irregularities is counted, if the quantity of the dump truck of irregularity is more than
When accounting of the dump truck of quantity monitoring threshold value and/or irregularity in whole dump trucks is more than ratio monitoring threshold value, to monitoring
System issues warning signal, and a large amount of irregularity vehicles occurs in warning, to realize the prompt of environmental pollution risk.
In one embodiment of the invention, it is preferable that as shown in fig. 6, the main flow of the monitoring method of target vehicle
Include:
S602 obtains history video data；
S604 obtains the band of position of service rack by history video data, and service rack is marked in monitor video
The band of position；
S606 reads monitor video in real time；
S608 detects dump truck in monitor video；
S610 judges it is then to enter S612 with the presence or absence of dump truck in monitor video, otherwise returns to S604；
S612 tracks dump truck position；
S614 records the residence time of dump truck when within the scope of the band of position that dump truck is in service rack；
S616, judges whether the residence time meets environmental protection tests specification, is then to enter S618, otherwise enters S620；
S618, statistics meet the dump truck quantity of specification；
S620, statistics do not meet the dump truck quantity of specification；
S622, the dump truck that calculating does not meet specification account for the ratio of all dump truck quantity；
S624, judgement do not meet whether ratio shared by the dump truck of specification is more than monitoring threshold value, are then to enter S626, no
Then return to S604；
S626 is issued warning signal to monitoring system.
In this embodiment, the band of position occurred in video by history monitor video data acquisition service rack, mark
Remember the band of position and obtains the coordinate information of the band of position；Service rack is set in real time monitoring video according to coordinate information
Band of position range.By whether including dump truck in target vehicle detection model inspection current monitor video, when including mud
When head vehicle, the band of position of the dump truck is tracked, when dump truck rests within the scope of the band of position of service rack, judges the mud
Whether the residence time of head vehicle meets environmental protection tests specification, and the dump truck is standardized or is not inconsistent labeled as meeting according to judging result
Close the dump truck of specification；Statistics meets specification and does not meet the quantity of the dump truck of specification respectively, calculates the mud for not meeting specification
Accounting of the head vehicle in all dump trucks, when not meeting the accounting of dump truck of specification is more than to monitor threshold value, to monitoring system
It issues warning signal.
As shown in fig. 7, a kind of monitoring system 700 of target vehicle is provided in the embodiment of second aspect of the present invention,
Including memory 702 and processor 704；Memory 702 for storing computer program, processor 704 by execute it is described based on
Calculation machine program with: by video input to be detected to target vehicle detection model, in each frame image to confirm video to be detected
It whether include target vehicle；When in video to be detected including target vehicle, obtained by the detection block of target vehicle detection model
Take the band of position of target vehicle；Tracing positional region；Judge whether the band of position is within the scope of target area；When position area
When domain is within the scope of target area, the stay time passed through when the band of position is within the scope of target area is obtained；Work as stop
When duration is less than duration threshold value, target vehicle is labeled as irregularity vehicle.
In this embodiment, it obtains video to be detected and is input to target vehicle detection model, first determine whether view to be detected
Whether there is target vehicle in frequency, after target vehicle occurs, is marked by detection block and obtain position locating for target vehicle
Set region, the band of position of the real-time tracking target vehicle, until the band of position is completely within the scope of target area, i.e. target
When vehicle is located within service rack, timing is opened to obtain target vehicle and is located at the stay time passed through in service rack, if
Stay time is less than duration threshold value, then shows the pollution that target vehicle cleans the sufficiently long time without service rack, on vehicle body
Object is unable to get effective cleaning, therefore the target vehicle is labeled as irregularity vehicle, and turn in a report to monitoring system.It answers
With technical solution provided by the invention, the dump truck in monitor video is accurately identified by target vehicle detection model, and
The position for tracking dump truck, judges whether the position of dump truck is in the position range of service rack, and is in and washes in dump truck
The residence time that dump truck is recorded when within the scope of ride realizes to whether dump truck meets the automatic, real of monitoring of environment requirement
When detection, ensure that the accuracy of the detection and tracking of dump truck, realize the intelligence of urban transportation and monitoring of environment.
In one embodiment of the invention, it is preferable that processor is also used to: modification target detection model is to be modified
Target detection model afterwards；Monitor video is acquired, the frame image comprising target vehicle is obtained by monitor video；Use detection block
The characteristic area of target vehicle in frame image is labeled, to form training dataset；It is repaired by training dataset training
Target detection model after changing, to obtain target vehicle detection model.
In this embodiment, general target detection model is obtained, it is preferable that select SSD (single shot
Multibox detection) model modifies as target detection model, and to it, to obtain modified detection mould
Type.A large amount of acquisition monitor video data, extracting in monitor video includes target vehicle, i.e. the frame image of dump truck, passes through modification
The detection block of detection model afterwards is labeled the characteristic area of the dump truck of frame image, specifically, can be by dump truck
Headstock region is as characteristic area and is labeled, and passes through the frame image making training dataset marked, wherein marked
The quantity of frame image, which is maintained at 20000 or more, can be obtained preferable effect, after being modified by the training dataset training made
Detection model, to obtain target vehicle detection model.The target detection model that training obtains according to the technical solution of the present invention
Have the characteristics that recognition accuracy is high, the false dismissal probability to the dump truck mutually blocked can be effectively reduced.
In one embodiment of the invention, it is preferable that processor is also used to: choosing target detection model, examined in target
The convolutional neural networks end for surveying model increases at least two layers symmetrical warp lamination；The non-maximum of target detection model is pressed down
Function formula modification processed are as follows:
Wherein, si is the confidence level of detection block, iou (M, bi) be detection block friendship and ratio, NpAnd NqFor for determining iou
(M,bi) value range constant.
In this embodiment, in the convolutional neural networks end of the general target detection model of selection, i.e. CNN
(Convolutional Neural Networks) at least two layers of network end-point increase symmetrical warp lamination, it is preferable that increase
3 layers of symmetrical warp lamination, and modify former non-maxima suppression function NMS (Non-maximum suppression):
Are as follows:
Wherein, si is the confidence level of detection block, iou (M, bi) be detection block friendship and ratio, NpAnd NqFor for determining iou
(M,bi) value range constant, compared to former non-maxima suppression function, modified non-maxima suppression function is increased
Threshold value, and reset according to the threshold value and reduce confidence level as original confidence levelRather than it is set to 0.It is repaired by above-mentioned
Changing can be effectively avoided is inhibited directly apart from target vehicle that is very close or mutually blocking between each other, and then improves and know
Other accuracy rate reduces the false dismissal probability to the dump truck mutually blocked.
In one embodiment of the invention, it is preferable that processor is also used to: KCF algorithm keeps track target vehicle is used, with
Obtain the real-time band of position of target vehicle.
In this embodiment, using the real time position of KCF algorithm keeps track target vehicle, calculation amount is small, and tracking result is accurate,
It can effectively improve dump truck position is accurately tracked, and then ensure that the band of position ground detection and tracking of dump truck
Accuracy.
In one embodiment of the invention, it is preferable that processor is also used to: the label target region in video to be detected
Range, to obtain the coordinate information of target area range；Obtain the center point coordinate of the band of position；When center point coordinate meets in advance
If when formula, determining that the band of position is within the scope of target area.
In this embodiment, after getting video to be detected, the band of position of service rack is marked out in detection video,
As target area range, and obtain the coordinate information in target zone region.After getting the band of position of target vehicle, meter
The center point coordinate for calculating the band of position, judges position by the coordinate information of preset formula, center point coordinate and preset range region
Set whether region is within the scope of target area, i.e., whether dump truck enters service rack, and then realizes and whether washing to dump truck
Stay long enough is judged in ride, and marks irregularity vehicle of the stay time less than duration threshold value.
In one embodiment of the invention, it is preferable that be obtained by the following formula the center point coordinate of target vehicle:
Wherein, coordinate, (x are put centered on (x, y)p,yp) be the band of position upper left corner coordinate, w be the band of position width
Degree, h are the height of the band of position.
In this embodiment, using the current frame image of target vehicle detection model inspection video to be detected, in present frame
There are the coordinate informations for when dump truck, exporting all dump truck headstock positions in picture detected in image.Specifically, false
If detecting M platform dump truck altogether, wherein the coordinate of the corresponding detection block of i-th dump truck is ci(xi,yi,wi,hi)；Wherein (xi,
yi) respectively indicate detection block the point of top left co-ordinate in the picture position, w and h respectively indicate the width of detection block in the picture
Degree and height.By tracking box track i-th dump truck detection block coordinate, and by above-mentioned formula calculating detection block or
The center point coordinate (x, y) of tracking box is sentenced when the center point coordinate (x, y) of detection block or tracking box meets preset formula
Location area is within the scope of target area, i.e., whether dump truck enters service rack.
In one embodiment of the invention, it is preferable that preset formula specifically:
Wherein, coordinate, (x are put centered on (x, y)min,ymin) be target area upper left angle point coordinate, wdFor target area
Width, hdFor the height of target area.
In this embodiment, the coordinate (x of the upper left angle point of the target area range of mark is obtainedmin,ymin), and further
Obtain the width w of target areadWith the height h of target aread, when center point coordinate (x, y) meets above-mentioned formula, decision bits
It sets region to be within the scope of target area, i.e., whether dump truck enters service rack.
In one embodiment of the invention, it is preferable that processor is also used to: when center point coordinate starts to meet default public affairs
When formula, record current point in time is first time point；At interval of default frame image, it is pre- that repetition judges whether center point coordinate meets
If formula；When center point coordinate starts to be unsatisfactory for preset formula, record current point in time was the second time point；When by first
Between point and the second time point calculate stay time.
In this embodiment, when center point coordinate starts to meet preset formula, illustrate that target vehicle enters target area
Domain range, i.e. dump truck enter service rack, and record current point in time is first time point lmin, at interval of default frame image, weight
Judge whether center point coordinate meets preset formula again, it is preferable that repeat to judge whether center point coordinate meets at interval of 30 frames
Preset formula is primary, reduces in the case where guaranteeing Detection accuracy and calculates pressure, when center point coordinate starts to be unsatisfactory for presetting
When formula, illustrate that target vehicle has left target area range, i.e. dump truck has left service rack, records current point in time conduct
Second time point lmax, the stay time of target vehicle can be calculated by first time point and the second time point, and then judging should
Whether dump truck is irregularity vehicle.
In one embodiment of the invention, it is preferable that stay time is calculated by first time point and the second time point
Formula, specifically:
T=lmax-lmin；
Wherein, T is stay time, lmaxFor the second time point, lminFor first time point.
In this embodiment, target vehicle leaves the time point of target area range and target vehicle enters target area model
Elapsed time section between the time point enclosed, i.e. stay time of the dump truck in service rack, can be with by calculating stay time
Judge whether current dump truck is irregularity vehicle.
In one embodiment of the invention, it is preferable that processor is also used to: the quantity of statistics irregularity vehicle；When not
It is more than ratio that the quantity for closing rule vehicle, which is more than the accounting of quantity monitoring threshold value and/or irregularity vehicle in target complete vehicle,
When monitoring threshold value, issue warning signal.
In this embodiment, the quantity of the dump truck of all irregularities is counted, if the quantity of the dump truck of irregularity is more than
When accounting of the dump truck of quantity monitoring threshold value and/or irregularity in whole dump trucks is more than ratio monitoring threshold value, to monitoring
System issues warning signal, and a large amount of irregularity vehicles occurs in warning, to realize the prompt of environmental pollution risk.
In the embodiment of third aspect present invention, a kind of computer readable storage medium is provided, is stored thereon with meter
Calculation machine program realizes the monitoring side of the target vehicle as described in above-mentioned any embodiment when computer program is executed by processor
Method, therefore the computer readable storage medium includes the whole of the monitoring method of any of the above-described target vehicle as described in the examples
Beneficial effect.
In the description of the present invention, term " multiple " then refers to two or more, unless otherwise restricted clearly, term
The orientation or positional relationship of the instructions such as "upper", "lower" is to be based on the orientation or positional relationship shown in the drawings, and is merely for convenience of retouching
It states the present invention and simplifies description, rather than the device or element of indication or suggestion meaning must have a particular orientation, with specific
Orientation construction and operation, therefore be not considered as limiting the invention；Term " connection ", " installation ", " fixation " etc. should all
It is interpreted broadly, for example, " connection " may be fixed connection or may be dismantle connection, or integral connection；It can be straight
Connect it is connected, can also be indirectly connected through an intermediary.It for the ordinary skill in the art, can be according to specific feelings
Condition understands the concrete meaning of above-mentioned term in the present invention.
In the description of the present invention, the description meaning of term " one embodiment ", " some embodiments ", " specific embodiment " etc.
Refer to that particular features, structures, materials, or characteristics described in conjunction with this embodiment or example are contained at least one implementation of the invention
In example or example.In the present invention, schematic expression of the above terms are not necessarily referring to identical embodiment or example.And
And the particular features, structures, materials, or characteristics of description can be in any one or more of the embodiments or examples with suitable
Mode combines.
The foregoing is only a preferred embodiment of the present invention, is not intended to restrict the invention, for the skill of this field
For art personnel, the invention may be variously modified and varied.All within the spirits and principles of the present invention, made any to repair
Change, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.
Claims (21)
1. a kind of monitoring method of target vehicle characterized by comprising
By video input to be detected to target vehicle detection model, in each frame image to confirm the video to be detected whether
Include target vehicle；
When in the video to be detected including the target vehicle, obtained by the detection block of the target vehicle detection model
The band of position of the target vehicle；
Track the band of position；
Judge whether the band of position is within the scope of target area；
When the band of position is within the scope of the target area, obtains the band of position and be in the target area model
The stay time passed through when enclosing interior；
When the stay time is less than duration threshold value, the target vehicle is labeled as irregularity vehicle.
2. the monitoring method of target vehicle according to claim 1, which is characterized in that described by video input to be detected
Before the step of to target vehicle detection model, further includes:
Target detection model is modified to obtain modified target detection model；
Monitor video is acquired, the frame image comprising the target vehicle is obtained by the monitor video；
It is labeled using characteristic area of the detection block to the target vehicle in the frame image, to form trained number
According to collection；
By the training dataset training modified target detection model, to obtain the target vehicle detection mould
Type.
3. the monitoring method of target vehicle according to claim 2, which is characterized in that the modification target detection model with
The step of obtaining modified target detection model, specifically includes:
Target detection model is chosen, increases at least two layers symmetrically instead in the convolutional neural networks end of the target detection model
Convolutional layer；
The non-maxima suppression function formula of the target detection model is modified are as follows:
Wherein, siFor the confidence level of the detection block, iou (M, bi) be the detection block friendship and ratio, NpAnd NqFor for determining
iou(M,bi) value range constant.
4. the monitoring method of target vehicle according to claim 1, which is characterized in that the tracking band of position
Step, specifically:
Using target vehicle described in KCF algorithm keeps track, to obtain the real-time band of position of the target vehicle.
5. the monitoring method of target vehicle according to claim 4, which is characterized in that
It is described judge the step whether band of position is within the scope of target area before, further includes:
The target area range is marked, in the video to be detected to obtain the coordinate information of the target area range；
And
The step for judging the band of position and whether being within the scope of target area, specifically includes:
Obtain the center point coordinate of the band of position；
When the center point coordinate meets preset formula, determine that the band of position is within the scope of the target area.
6. the monitoring method of target vehicle according to claim 5, which is characterized in that be obtained by the following formula the mesh
Mark the center point coordinate of vehicle:
Wherein, (x, y) is the center point coordinate, (xp,yp) be the band of position upper left corner coordinate, w be the position area
The width in domain, h are the height of the band of position.
7. the monitoring method of target vehicle according to claim 5, which is characterized in that the preset formula specifically:
Wherein, (x, y) is the center point coordinate, (xmin,ymin) be target area upper left angle point coordinate, wdIt is described
The width of target area, hdFor the height of the target area.
8. the monitoring method of target vehicle according to claim 5, which is characterized in that described to obtain at the band of position
It the step of stay time passed through when within the scope of the target area, specifically includes:
When the center point coordinate starts to meet the preset formula, record current point in time is first time point；
At interval of default frame image, repetition judges whether the center point coordinate meets the preset formula；
When the center point coordinate starts to be unsatisfactory for the preset formula, record current point in time was the second time point；
The stay time is calculated by the first time point and second time point.
9. the monitoring method of target vehicle according to claim 8, which is characterized in that described to pass through the first time point
The formula of the stay time is calculated with second time point, specifically:
T=lmax-lmin；
Wherein, T is the stay time, lmaxFor second time point, lminFor the first time point.
10. the monitoring method of target vehicle according to any one of claim 1 to 9, which is characterized in that further include:
Count the quantity of the irregularity vehicle；
When the quantity of the irregularity vehicle is more than quantity monitoring threshold value and/or the irregularity vehicle in all targets
When accounting in vehicle is more than ratio monitoring threshold value, issue warning signal.
11. a kind of monitoring system of target vehicle, which is characterized in that including
Memory, for storing computer program；
Processor, for execute the computer program with:
By video input to be detected to target vehicle detection model, in each frame image to confirm the video to be detected whether
Include target vehicle；
When in the video to be detected including the target vehicle, obtained by the detection block of the target vehicle detection model
The band of position of the target vehicle；
Track the band of position；
Judge whether the band of position is within the scope of target area；
When the band of position is within the scope of the target area, obtains the band of position and be in the target area model
The stay time passed through when enclosing interior；
When the stay time is less than duration threshold value, the target vehicle is labeled as irregularity vehicle.
12. the monitoring system of target vehicle according to claim 11, which is characterized in that the processor is also used to:
Target detection model is modified to obtain modified target detection model；
Monitor video is acquired, the frame image comprising the target vehicle is obtained by the monitor video；
It is labeled using characteristic area of the detection block to the target vehicle in the frame image, to form trained number
According to collection；
By the training dataset training modified target detection model, to obtain the target vehicle detection mould
Type.
13. the monitoring system of target vehicle according to claim 12, which is characterized in that the processor is also used to:
Target detection model is chosen, increases at least two layers symmetrically instead in the convolutional neural networks end of the target detection model
Convolutional layer；
The non-maxima suppression function formula of the target detection model is modified are as follows:
Wherein, siFor the confidence level of the detection block, iou (M, bi) be the detection block friendship and ratio, NpAnd NqFor for determining
iou(M,bi) value range constant.
14. the monitoring system of target vehicle according to claim 12, which is characterized in that the processor is also used to:
Using target vehicle described in KCF algorithm keeps track, to obtain the real-time band of position of the target vehicle.
15. the monitoring system of target vehicle according to claim 14, which is characterized in that the processor is also used to:
The target area range is marked, in the video to be detected to obtain the coordinate information of the target area range；
And
Obtain the center point coordinate of the band of position；
When the center point coordinate meets preset formula, determine that the band of position is within the scope of the target area.
16. the monitoring system of target vehicle according to claim 15, which is characterized in that be obtained by the following formula described
The center point coordinate of target vehicle:
Wherein, (x, y) is the center point coordinate, (xp,yp) be the band of position top left co-ordinate, w be the position area
The width in domain, h are the height of the band of position.
17. the monitoring system of target vehicle according to claim 15, which is characterized in that the preset formula specifically:
Wherein, (x, y) is the center point coordinate, (xmin,ymin) be target area upper left angle point coordinate, wdIt is described
The width of target area, hdFor the height of the target area.
18. the monitoring system of target vehicle according to claim 15, which is characterized in that the processor is also used to:
When the center point coordinate starts to meet the preset formula, record current point in time is first time point；
At interval of default frame image, repetition judges whether the center point coordinate meets the preset formula；
When the center point coordinate starts to be unsatisfactory for the preset formula, record current point in time was the second time point；
The stay time is calculated by the first time point and second time point.
19. the monitoring system of target vehicle according to claim 18, which is characterized in that the first time point and described
Second time point calculated the formula of the stay time, specifically:
T=lmax-lmin；
Wherein, T is the stay time, lmaxFor second time point, lminFor the first time point.
20. the monitoring system of target vehicle described in any one of 1 to 19 according to claim 1, which is characterized in that the processing
Device is also used to:
Count the quantity of the irregularity vehicle；
When the quantity of the irregularity vehicle is more than quantity monitoring threshold value and/or the irregularity vehicle in all targets
When accounting in vehicle is more than ratio monitoring threshold value, issue warning signal.
21. a kind of computer readable storage medium, is stored thereon with computer program, it is characterised in that: the computer program
The monitoring method of the target vehicle as described in any one of claims 1 to 10 is realized when being executed by processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910020540.9A CN109800696A (en) | 2019-01-09 | 2019-01-09 | Monitoring method, system and the computer readable storage medium of target vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910020540.9A CN109800696A (en) | 2019-01-09 | 2019-01-09 | Monitoring method, system and the computer readable storage medium of target vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109800696A true CN109800696A (en) | 2019-05-24 |
Family
ID=66557060
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910020540.9A Pending CN109800696A (en) | 2019-01-09 | 2019-01-09 | Monitoring method, system and the computer readable storage medium of target vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109800696A (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110324583A (en) * | 2019-07-15 | 2019-10-11 | 深圳中兴网信科技有限公司 | A kind of video monitoring method, video monitoring apparatus and computer readable storage medium |
| CN110688902A (en) * | 2019-08-30 | 2020-01-14 | 智慧互通科技有限公司 | Method and device for detecting vehicle area in parking space |
| CN110852212A (en) * | 2019-10-29 | 2020-02-28 | 上海眼控科技股份有限公司 | Method and device for checking operation object in vehicle detection |
| CN111126397A (en) * | 2019-12-27 | 2020-05-08 | 上海眼控科技股份有限公司 | Dynamic inspection method, system, terminal and medium for vehicle chassis |
| CN111242042A (en) * | 2020-01-15 | 2020-06-05 | 上海眼控科技股份有限公司 | Inspection duration judgment method, apparatus, device and medium based on vehicle detection |
| CN111414833A (en) * | 2020-03-16 | 2020-07-14 | 北京嘀嘀无限科技发展有限公司 | Target detection tracking method, target detection tracking device, and storage medium |
| CN112101202A (en) * | 2020-09-14 | 2020-12-18 | 北京易华录信息技术股份有限公司 | Construction scene recognition method and device and electronic equipment |
| CN112131966A (en) * | 2020-09-01 | 2020-12-25 | 深圳中兴网信科技有限公司 | Mud truck monitoring method and system and storage medium |
| CN112419733A (en) * | 2020-10-16 | 2021-02-26 | 杭州智行星科技有限公司 | Method, device, equipment and storage medium for detecting irregular parking of user |
| CN112509315A (en) * | 2020-11-04 | 2021-03-16 | 杭州远眺科技有限公司 | Traffic accident detection method based on video analysis |
| CN112507813A (en) * | 2020-11-23 | 2021-03-16 | 北京旷视科技有限公司 | Event detection method and device, electronic equipment and storage medium |
| CN112530176A (en) * | 2020-11-10 | 2021-03-19 | 浙江大华系统工程有限公司 | Drunk driving early warning method, device, equipment and medium |
| CN112822458A (en) * | 2021-01-12 | 2021-05-18 | 南京易自助网络科技有限公司 | Video monitoring method and terminal for car washing process |
| CN112861585A (en) * | 2019-11-27 | 2021-05-28 | 杭州海康威视数字技术股份有限公司 | Vehicle washing detection method and device and electronic equipment |
| CN113468974A (en) * | 2021-06-08 | 2021-10-01 | 深圳依时货拉拉科技有限公司 | Traffic flow statistical method, computer readable storage medium and mobile terminal |
| CN114639037A (en) * | 2022-03-03 | 2022-06-17 | 青岛海信网络科技股份有限公司 | Method for determining vehicle saturation of high-speed service area and electronic equipment |
| WO2023179416A1 (en) * | 2022-03-22 | 2023-09-28 | 阿里云计算有限公司 | Method and apparatus for determining entry and exit of vehicle into and out of parking space, device, and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102411767A (en) * | 2011-11-25 | 2012-04-11 | 中科怡海高新技术发展江苏股份公司 | Construction site intelligent monitoring system and method based on internet of things |
| CN203974768U (en) * | 2014-06-27 | 2014-12-03 | 宝山钢铁股份有限公司 | A kind of stock ground service rack is responded to and washing equipment automatically with vehicle |
| CN108229475A (en) * | 2018-01-03 | 2018-06-29 | 深圳中兴网信科技有限公司 | Wireless vehicle tracking, system, computer equipment and readable storage medium storing program for executing |
| CN108875595A (en) * | 2018-05-29 | 2018-11-23 | 重庆大学 | A kind of Driving Scene object detection method merged based on deep learning and multilayer feature |
| CN109117702A (en) * | 2018-06-12 | 2019-01-01 | 深圳中兴网信科技有限公司 | The detection and count tracking method and system of target vehicle |
-
2019
- 2019-01-09 CN CN201910020540.9A patent/CN109800696A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102411767A (en) * | 2011-11-25 | 2012-04-11 | 中科怡海高新技术发展江苏股份公司 | Construction site intelligent monitoring system and method based on internet of things |
| CN203974768U (en) * | 2014-06-27 | 2014-12-03 | 宝山钢铁股份有限公司 | A kind of stock ground service rack is responded to and washing equipment automatically with vehicle |
| CN108229475A (en) * | 2018-01-03 | 2018-06-29 | 深圳中兴网信科技有限公司 | Wireless vehicle tracking, system, computer equipment and readable storage medium storing program for executing |
| CN108875595A (en) * | 2018-05-29 | 2018-11-23 | 重庆大学 | A kind of Driving Scene object detection method merged based on deep learning and multilayer feature |
| CN109117702A (en) * | 2018-06-12 | 2019-01-01 | 深圳中兴网信科技有限公司 | The detection and count tracking method and system of target vehicle |
Non-Patent Citations (2)
| Title |
|---|
| 孙皓泽;常天庆;王全东;孔德鹏;戴文君;: "一种基于分层多尺度卷积特征提取的坦克装甲目标图像检测方法" * |
| 尹勰;闫磊;: "基于深度卷积神经网络的图像目标检测" * |
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110324583A (en) * | 2019-07-15 | 2019-10-11 | 深圳中兴网信科技有限公司 | A kind of video monitoring method, video monitoring apparatus and computer readable storage medium |
| CN110688902A (en) * | 2019-08-30 | 2020-01-14 | 智慧互通科技有限公司 | Method and device for detecting vehicle area in parking space |
| CN110688902B (en) * | 2019-08-30 | 2022-02-11 | 智慧互通科技股份有限公司 | Method and device for detecting vehicle area in parking space |
| CN110852212A (en) * | 2019-10-29 | 2020-02-28 | 上海眼控科技股份有限公司 | Method and device for checking operation object in vehicle detection |
| CN112861585B (en) * | 2019-11-27 | 2024-05-28 | 杭州海康威视数字技术股份有限公司 | Vehicle washing detection method and device and electronic equipment |
| CN112861585A (en) * | 2019-11-27 | 2021-05-28 | 杭州海康威视数字技术股份有限公司 | Vehicle washing detection method and device and electronic equipment |
| CN111126397A (en) * | 2019-12-27 | 2020-05-08 | 上海眼控科技股份有限公司 | Dynamic inspection method, system, terminal and medium for vehicle chassis |
| CN111242042A (en) * | 2020-01-15 | 2020-06-05 | 上海眼控科技股份有限公司 | Inspection duration judgment method, apparatus, device and medium based on vehicle detection |
| CN111414833A (en) * | 2020-03-16 | 2020-07-14 | 北京嘀嘀无限科技发展有限公司 | Target detection tracking method, target detection tracking device, and storage medium |
| CN112131966A (en) * | 2020-09-01 | 2020-12-25 | 深圳中兴网信科技有限公司 | Mud truck monitoring method and system and storage medium |
| CN112101202A (en) * | 2020-09-14 | 2020-12-18 | 北京易华录信息技术股份有限公司 | Construction scene recognition method and device and electronic equipment |
| CN112419733A (en) * | 2020-10-16 | 2021-02-26 | 杭州智行星科技有限公司 | Method, device, equipment and storage medium for detecting irregular parking of user |
| CN112509315A (en) * | 2020-11-04 | 2021-03-16 | 杭州远眺科技有限公司 | Traffic accident detection method based on video analysis |
| CN112530176A (en) * | 2020-11-10 | 2021-03-19 | 浙江大华系统工程有限公司 | Drunk driving early warning method, device, equipment and medium |
| CN112530176B (en) * | 2020-11-10 | 2022-03-25 | 浙江大华系统工程有限公司 | Drunk driving early warning method, device, equipment and medium |
| WO2022105243A1 (en) * | 2020-11-23 | 2022-05-27 | 北京旷视科技有限公司 | Event detection method, apparatus, electronic device, and storage medium |
| CN112507813A (en) * | 2020-11-23 | 2021-03-16 | 北京旷视科技有限公司 | Event detection method and device, electronic equipment and storage medium |
| CN112822458B (en) * | 2021-01-12 | 2022-05-13 | 南京易自助网络科技有限公司 | Video monitoring method and terminal for car washing process |
| CN112822458A (en) * | 2021-01-12 | 2021-05-18 | 南京易自助网络科技有限公司 | Video monitoring method and terminal for car washing process |
| CN113468974A (en) * | 2021-06-08 | 2021-10-01 | 深圳依时货拉拉科技有限公司 | Traffic flow statistical method, computer readable storage medium and mobile terminal |
| CN113468974B (en) * | 2021-06-08 | 2024-04-19 | 深圳依时货拉拉科技有限公司 | Traffic flow statistics method, computer readable storage medium and mobile terminal |
| CN114639037A (en) * | 2022-03-03 | 2022-06-17 | 青岛海信网络科技股份有限公司 | Method for determining vehicle saturation of high-speed service area and electronic equipment |
| CN114639037B (en) * | 2022-03-03 | 2024-04-09 | 青岛海信网络科技股份有限公司 | Method for determining vehicle saturation of high-speed service area and electronic equipment |
| WO2023179416A1 (en) * | 2022-03-22 | 2023-09-28 | 阿里云计算有限公司 | Method and apparatus for determining entry and exit of vehicle into and out of parking space, device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109800696A (en) | Monitoring method, system and the computer readable storage medium of target vehicle | |
| CN110261436B (en) | Rail fault detection method and system based on infrared thermal imaging and computer vision | |
| CN104537841B (en) | Unlicensed vehicle violation detection method and detection system thereof | |
| CN105260713B (en) | A kind of method for detecting lane lines and device | |
| CN106228835B (en) | A kind of parking stall Parking judgment method and system | |
| CN103927878B (en) | A kind of automatic shooting device for parking offense and automatically grasp shoot method | |
| CN105809975A (en) | Abnormal parking judgment method and abnormal parking judgment device | |
| RU2698610C2 (en) | Method and processing unit for control over traffic monitoring system | |
| KR100969995B1 (en) | System of traffic conflict decision for signalized intersections using image processing technique | |
| KR101908611B1 (en) | Parking control method for vehicle | |
| CN100454355C (en) | Video method for collecting information of vehicle flowrate on road in real time | |
| CN107886055A (en) | A kind of retrograde detection method judged for direction of vehicle movement | |
| CN108665700B (en) | Method and device for detecting lateral parking state | |
| CN103366569A (en) | Method and system for snapshotting traffic violation vehicle in real time | |
| CN106778593A (en) | A kind of track level localization method based on the fusion of many surface marks | |
| CN103605967A (en) | Subway fare evasion prevention system and working method thereof based on image recognition | |
| CN109269478A (en) | A kind of container terminal based on binocular vision bridge obstacle detection method | |
| CN105741559A (en) | Emergency vehicle lane illegal occupation detection method based on lane line model | |
| CN101615342B (en) | Vehicle detection method | |
| CN108154146A (en) | A kind of car tracing method based on image identification | |
| CN107240268A (en) | A kind of detection method of high speed Emergency Vehicle Lane vehicles peccancy towards unmanned plane | |
| CN106710224A (en) | Evidence taking method and device for vehicle illegal driving | |
| CN105574502A (en) | Automatic detection method for violation behaviors of self-service card sender | |
| CN109117702A (en) | The detection and count tracking method and system of target vehicle | |
| CN111583659B (en) | Vehicle violation determination method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190524 |
|
| WD01 | Invention patent application deemed withdrawn after publication |