CN104954741A - Tramcar on-load and no-load state detecting method and system based on deep-level self-learning network - Google Patents
Tramcar on-load and no-load state detecting method and system based on deep-level self-learning network Download PDFInfo
- Publication number
- CN104954741A CN104954741A CN201510290352.XA CN201510290352A CN104954741A CN 104954741 A CN104954741 A CN 104954741A CN 201510290352 A CN201510290352 A CN 201510290352A CN 104954741 A CN104954741 A CN 104954741A
- Authority
- CN
- China
- Prior art keywords
- mine car
- module
- status image
- extreme value
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a tramcar on-load and no-load state detecting method and system based on the deep-level self-learning network. The tramcar on-load and no-load state detecting method includes steps of inputting various tramcar or non-tramcar state images and storing the same into a sample bank; deriving and expanding the sample bank; constructing the deep-level self-learning network to acquire monitoring images, analyzing and comparing the monitoring images according to the derived and expanded sample bank, comparing the monitoring images with the tramcar no-load and on-load state images stored in the sample bank, and storing the images into a tramcar no-load state image sample database, a tramcar on-load state image sample database or non-tramcar state image sample database in the sample bank according to comparison; storing the acquired images into corresponding databases in the derived and expanded sample bank. The tramcar on-load and no-load state detecting method has the advantages that on-load and no-load states of tramcars can be detected accurately by camera monitoring, manual intervention is reduced, and the tramcars can be automatically managed.
Description
Technical field
The present invention relates to machine learning method to be applied in mining site in mine car intelligent operation, realizes the Aulomatizeted Detect that mine car carries full two state of value of ore deposit sky.Be particularly suitable under condition complicated and changeable, comprise and there is intensity of illumination change, mine dust interference and mine car position arbitrarily change, needs can pass through camera head monitor exactly, detect that mine car carries the full state of sky in ore deposit, reduce manual intervention, realize the scene of mine car operation automation management.
Background technology
At present, realize intellectuality and the automatic management of mine operation, greatly can improve mining, the efficiency in fortune ore deposit.And the Aulomatizeted Detect of the wherein full state of mine car sky, accurate information can be provided for mine car scheduling, and then mine car scheduling can be optimized, improve the utilization ratio of mine car.But in the mining site operation of reality, the environmental condition residing for mine car is very complicated and changeable.The illumination variation that open work brings, the dimness of vision that mine dust brings, and the randomness that mine car is parked, all can disturb the video monitoring of mine car.How detecting that mine car carries the full state of sky in ore deposit exactly, is a challenging and significantly problem.
Adopt machine learning, detecting the full state of sky of mine car, is a kind of very effective detection method.Machine learning can carry out sample learning to various complex conditions, by study great amount of samples, can promote the generalization ability of learner and detect classification capacity, and then can detect the full state of the sky of mine car on high robust ground.In the process being applied to the empty slow state-detection of mine car, relate to the design of machine learning device.Design robustness, and the monitor and detection of learner to mine car with very strong generalization ability has very large meaning.With the different time sections of same mine car in different mining sites, the sight residing for mine car is different.Make a large amount of learning samples, need very high artificial mark cost, this is a no small problem to the intelligent operation of mine car.Therefore, the learner of design should have online ability of self-teaching, and the Sample Storehouse of renewal that can be real-time oneself, just can reduce manual intervention, thus very practicably meets the detection sight that mine car carries ore deposit state.
In the design and researchp of learner, the people such as Hinton are at " ImageNet Classification with Deep Convolutional Neural Networks ' (Neural Information Processing Systems 2012); specialized designs is based on the machine learning device of profound e-learning structure; by the mode of supervised learning, can realize very high object nicety of grading.They are based on the work of profound e-learning, and the design for the machine learning device of High Precision Robust opens a new visual angle.Subsequently, profound learning network comprises image recognition in many fields, and speech recognition and natural character word processing, achieve huge success.But this kind of machine learning device, needs a large amount of samples to carry out off-line training.Simultaneously the grader that generates of off-line training, cannot learn the situation of change of actual mine car monitoring scene online, and is only the generalization ability being strengthened study by the diversity of a large amount of training sample and otherness.In addition, this kind of machine learning method, from the base pixel of a large amount of static samples pictures, removes angle point in study image, the information such as edge, and then builds high-level semantic, have certain blindness in the study.Directly apply to the monitor video sequence with time dimension, cannot excavate on time dimension well, object to be detected or to be sorted in structure, the change in the information such as color.Therefore, for the monitor video sequence with high correlation, under mine intelligent equalization, realize the detection of the full state of mine car sky, need to design special profound e-learning structure, fully excavate the information on time dimension, and then improve the precision detecting classification.
In addition, as the elaboration above us, in order to improve the robustness detecting classification, profound learning network should have the ability of on-line study, in the process of mine car monitoring, can make correct study, exclusive PCR online to the change of around scene.In on-line study mechanism, the people such as Severin are in " Beyond Semi-Supervised Tracking:Tracking Should Be as Simple as Detection; but not Simpler than Recognition ", by on-line study candidate region and neighboring area, achieve the self-teaching of shallow-layer network.This patent carries the monitoring of the full state of ore deposit sky about mine car for mine car operation, set about emphatically from profound e-learning structural design and self-teaching mechanism these two aspects, proposes the full state robust detection method of mine car sky based on profound self-teaching.
Summary of the invention
The present invention is intended at least one of solve the problems of the technologies described above.
For this reason, first object of the present invention is the detection method proposing the full state of a kind of profound self-teaching real-time performance mine car sky.
Second object of the present invention is the detection system proposing the full state of a kind of profound self-teaching real-time performance mine car sky.
To achieve these goals, embodiments of the invention disclose the detection method of the full state of a kind of profound self-teaching real-time performance mine car sky, comprise the following steps: A. inputs multiple mine car dummy status image, the full status image of multiple mine car and multiple non-mine car status image and is stored as initial Sample Storehouse; B. derivative expansion is carried out to initial Sample Storehouse; C. build profound learning network to gather monitoring image, according to the described Sample Storehouse after derivative expansion, described monitoring image is analysed and compared, by the described mine car dummy status image stored in described monitoring image and described Sample Storehouse, the full status image of described mine car respectively with described mine car status image comparison difference, according to described comparison difference by described image stored in mine car dummy status image pattern storehouse, the full status image Sample Storehouse of mine car or non-mine car status image Sample Storehouse in described Sample Storehouse; And the described image gathered is carried out derivative expansion in the respective sample storehouse of described Sample Storehouse by D..
According to the detection method of the full state of a kind of profound self-teaching real-time performance mine car sky of the embodiment of the present invention, camera head monitor can be passed through exactly, detect that mine car carries the full state of sky in ore deposit, reduce manual intervention, realize the management of mine car operation automation.
In addition, the detection method of the full state of a kind of profound self-teaching real-time performance mine car sky according to the above embodiment of the present invention can also have following additional technical characteristic:
Further, in stepb, tell the derivative mode expanded mine car dummy status image, the full status image of mine car and the non-mine car status image comprised described Sample Storehouse and carry out affine transformation and/or noise and add and/or brightly to regulate.
Further, in step C, comprise further: the frame of video that C1. chooses N continuous frame is divided into three passages to carry out color displacement, shape displacement and monochrome information respectively extracting, wherein, color displacement refers to carry out difference frame by frame to the RGB passage of video frame pixel described in N frame, and the average of described RGB passage after asking for difference, shape displacement refers to the change in location obtaining the described video frame motion part of monitoring, and monochrome information refers to the gray value directly recording each frame of described frame of video; C2. the channel information of three described passages carried out convolution and ask for extreme value; C3. read described mine car dummy status image pattern storehouse, the full status image Sample Storehouse of described mine car and non-mine car status image Sample Storehouse and calculate the difference with described extreme value, by described frame of video stored in the full status image Sample Storehouse of described mine car and the minimum mine car dummy status image pattern storehouse of non-mine car status image Sample Storehouse difference value, the full status image Sample Storehouse of described mine car or non-mine car status image Sample Storehouse.
Further, in step C2, comprise further: the channel information of three described passages is carried out first time convolution ask for the first extreme value by C21.; C22. carry out second time convolution according to described first extreme value and ask for secondary extremal.
Further, in step C3, comprise further: C31. obtains described first extreme value and described secondary extremal; C32. SOFTMAX is adopted to calculate secondary extremal described in described first extreme value and described secondary extremal and described mine car dummy status image pattern storehouse, the full difference value between status image Sample Storehouse and non-mine car status image Sample Storehouse of described mine car, when described difference value is less than a preset value, by described frame of video stored in corresponding described Sample Storehouse.
Further, in step D, carried out by the described image gathered derivative expanding in the respective sample storehouse of described Sample Storehouse, the method for described derivative expansion comprises and rotates described image, tilts, introduces noise, regulates at least one item become clear and in adjustment contrast.
To achieve these goals, embodiments of the invention disclose the detection system of the full state of a kind of profound self-teaching real-time performance mine car sky, comprise Sample Storehouse, for storing multiple mine car dummy status images of initial input, the full status image of multiple mine car and multiple non-mine car status image; Derivative enlargement module, for carrying out derivative expansion to described mine car dummy status image, the full status image of described mine car and described non-mine car status image; Image capture module; And selection enlargement module, derive the described mine car dummy status image after expansion, the full status image of described mine car and described non-mine car status image and to compare with the image of described image capture module collection respectively for reading described derivative enlargement module and obtain difference value, and according to described difference value by the relevant position of described image stored in described Sample Storehouse.
According to the detection system of the full state of a kind of profound self-teaching real-time performance mine car sky of the embodiment of the present invention, camera head monitor can be passed through exactly, detect that mine car carries the full state of sky in ore deposit, reduce manual intervention, realize the management of mine car operation automation.
In addition, according to the detection system of the full state of a kind of profound self-teaching real-time performance mine car sky of the embodiment of the present invention, following additional technical characteristic can also be had:
Further, described selection enlargement module comprises further: read module, derives the described mine car dummy status image after expansion, the full status image of described mine car and described non-mine car status image for reading described derivative enlargement module; Segmentation extraction module, for being that three passages carry out color displacement respectively by the Iamge Segmentation of video acquisition module collection, the extraction of shape displacement and monochrome information, wherein, color displacement refers to carry out difference frame by frame to the RGB passage of N frame video frame pixel, and the average of described RGB passage after asking for difference, shape displacement refers to the change in location obtaining the described video frame motion part of monitoring, and monochrome information refers to the gray value directly recording each frame of described frame of video; Convolutional calculation module, carries out convolutional calculation for the channel information that receives from described segmentation extraction module; Extreme value asks for module, and for asking for extreme value to the result of calculation of convolutional calculation module, described extreme value comprises maximum and minimum; Sort module, described derivative enlargement module for reading according to described read module derives the described mine car dummy status image after expansion, the full status image of described mine car and described non-mine car status image and asks for ratio of extreme values that module asks for respectively with described extreme value to difference, and classifies to described image according to described difference; And enlargement module, for seeing that sorted described image extends in the relevant position of described Sample Storehouse.
Further, described convolutional calculation module comprises the first convolution computing module, the second convolution computing module, the 3rd convolutional calculation module and Volume Four and amasss computing module; Described extreme value is asked for module and is comprised that the first extreme value asks for module, secondary extremal asks for module, the 3rd extreme value asks for module and the 4th extreme value asks for module; Wherein, described first convolution computing module is for calculating the first volume product value of described channel information, and described first extreme value asks for module for asking for the first extreme value to described first volume product value; Described second convolution computing module is used for calculating volume Two product value according to described first extreme value, and described secondary extremal asks for module for asking for secondary extremal to described volume Two product value; Described 3rd convolutional calculation module is used for calculating the 3rd convolution value according to described secondary extremal, and described 3rd extreme value asks for module for asking for the 3rd extreme value to described 3rd convolution value; Computing module is amassed for calculating Volume Four product value according to described 3rd extreme value in described Volume Four, and described 4th extreme value asks for module for asking for the 4th extreme value to described Volume Four product value.
Further, described selection enlargement module also comprises full link block, described full link block is for receiving described first extreme value, secondary extremal, the 3rd extreme value and described 4th extreme value, and described first extreme value, secondary extremal, the 3rd extreme value and described 4th extreme value are passed to described sort module classify, described sort module adopts SOFTMAX regression model to classify.
Additional aspect of the present invention and advantage will part provide in the following description, and part will become obvious from the following description, or be recognized by practice of the present invention.
Accompanying drawing explanation
Above-mentioned and/or additional aspect of the present invention and advantage will become obvious and easy understand from accompanying drawing below combining to the description of embodiment, wherein:
Fig. 1 is the structural representation of profound learning network of the present invention;
Fig. 2 is structural representation of the present invention.
Embodiment
Be described below in detail embodiments of the invention, the example of described embodiment is shown in the drawings, and wherein same or similar label represents same or similar element or has element that is identical or similar functions from start to finish.Being exemplary below by the embodiment be described with reference to the drawings, only for explaining the present invention, and can not limitation of the present invention being interpreted as.
In describing the invention, it will be appreciated that, term " " center ", " longitudinal direction ", " transverse direction ", " on ", D score, " front ", " afterwards ", " left side ", " right side ", " vertically ", " level ", " top ", " end ", " interior ", orientation or the position relationship of the instruction such as " outward " are based on orientation shown in the drawings or position relationship, only the present invention for convenience of description and simplified characterization, instead of indicate or imply that the device of indication or element must have specific orientation, with specific azimuth configuration and operation, therefore limitation of the present invention can not be interpreted as.In addition, term " first ", " second " only for describing object, and can not be interpreted as instruction or hint relative importance.
In describing the invention, it should be noted that, unless otherwise clearly defined and limited, term " installation ", " being connected ", " connection " should be interpreted broadly, and such as, can be fixedly connected with, also can be removably connect, or connect integratedly; Can be mechanical connection, also can be electrical connection; Can be directly be connected, also indirectly can be connected by intermediary, can be the connection of two element internals.For the ordinary skill in the art, concrete condition above-mentioned term concrete meaning in the present invention can be understood.
With reference to description below and accompanying drawing, these and other aspects of embodiments of the invention will be known.Describe at these and in accompanying drawing, specifically disclose some particular implementation in embodiments of the invention, representing some modes of the principle implementing embodiments of the invention, but should be appreciated that the scope of embodiments of the invention is not limited.On the contrary, embodiments of the invention comprise fall into attached claims spirit and intension within the scope of all changes, amendment and equivalent.
Describe according to embodiments of the invention below in conjunction with accompanying drawing.
Fig. 1 is the structural representation of profound learning network of the present invention, and Fig. 2 is structural representation of the present invention.Please refer to Fig. 1 and Fig. 2.
1, profound learning network pre-training
(1) mine car loading status image sample prepares
All sample standard deviations derive from mine supervision camera, to the video acquisition of mine car operation loading state.When doing initial pre-training, by artificial mark, be divided three classes by sample, first kind digital label is expressed as 0, represents mine car current state for empty, does not namely have loading.Equations of The Second Kind digital label is expressed as 1, represents that mine car current state is full, has namely carried full goods.3rd class digital label is expressed as 2, represents other samples of non-mine car state.In order to ensure the diversity of sample, strengthen generalization ability, by day, at night, at the cloudy day, fine day, carries out under the conditions such as rainy day collecting sample, gathers the Different categories of samples of identical number under various condition.In training sample, to represent mine car state be empty number of samples is 100,000, and to represent mine current state be full sample number is also 100,000, and what represent non-mine car state is 20,000.In checking sample, to represent mine car state be empty number of samples is 1,000, and to represent mine current state be full sample number is also 1,000, represents the sample 2,000 of non-mine car state.Wherein digital label is the sample of 0 or 1, also refers to positive sample above-mentioned.And digital label is the sample of 2, also refer to negative sample above-mentioned.
(2) the profound learning network built, to the sample training of demarcating in advance
For mine car monitor video, design profound learning network, be intended to the feature representation obtaining the full state of mine car sky more exactly.Profound network has from pixel, to edge, then arrives the accurate ability to express of high-rise shape semanteme.Profound learning network of the present invention comprises a dividing layer, 4 convolutional layers, 4 extreme value layers, 1 full interconnect layer and 1 three grader based on SOFTMAX regression model.
To input mine car monitor video frame sequence carry out training type, sequence of frames of video sample elementary cell number we get 10.Frame of video training sample is divided into three passages, carries out color displacement respectively, the extraction of shape displacement and monochrome information.
Wherein color displacement refers to and carries out difference frame by frame respectively to the RGB passage of video frame pixel, and the average of RGB passage after asking for difference.
And shape displacement refers to the change in location obtaining monitoring video frame motion parts, the assignment that changes is 1, and unchanged is 0.
And monochrome information refers to the gray value of each frame of direct recording of video frame.
For mine car monitor video sequence, obtain color displacement, shape displacement and monochrome information, fully can excavate the space structure between frame and frame and color, brightness variation relation, thus the dependency relation that make use of interframe fully.Therefore the state change information of mine car under monitor video can be described more exactly, for profound e-learning provides information source more fully.
For three channel informations extracted, dividing layer completes three layers, the extraction of positive and negative sample areas.Wherein positive sample comprises the image-region of car completely and under empty two states of car, and negative sample is the image-region of non-mine car.
Through dividing layer, information enters convolutional layer 1, and the 3D convolution kernel based on 10 class 3x3x10 completes convolution.
After convolution, information enters extreme value layer 1.This layer adopts 3x3 template to ask for extreme value.Extreme value comprises maximum and minimum.
After this, the data of extreme value layer 1 flow into convolutional layer 2, then complete convolution based on the 3D convolution kernel of 10 class 3x3x10.
The data of convolutional layer 2 flow into extreme value layer 2, in like manner, complete based on 3x3 template asking for image-region extreme value.
The data of extreme value layer 2 flow into convolutional layer 3, and at this layer, the 4x4 convolution kernel of base 10 class 2D, completes the convolution to extreme value layer data.
This layer data flows into extreme value layer 3, in like manner completes and asks for image extreme value based on 3x3 template.
The data of extreme value layer 3 enter convolutional layer 4, and at this layer, the 5x5 convolution kernel of base 10 class 2D completes the convolution to extreme value layer data.
Data after convolution flow into extreme value layer 4, complete and ask for based on the extreme value of 3x3 template to image-region.
Extreme value data finally flow into full articulamentum.Full articulamentum completes the full expand to extreme value layer 4 data, accepts from extreme value layer 1,2, the data of 3 simultaneously.The data length receiving extreme value layer 1,2,3 is fixing, equals the data scale that extreme value layer 4 launches.Adopt the random mode received simultaneously.Adopt the random mode received, full articulamentum carries out feature representation ability to the different phase of profound learning network can be strengthened.The data that full articulamentum receives extreme value layer 1,2,3,4 are also named receptive field transmission, mean and the impression information of different layers are delivered on full articulamentum.Catch up with three graders of a SOFTMAX after Quan Lian basic unit, complete mine car empty, the detection of full and non-mine car state.
SOFTMAX grader judges that mine car is empty, full and non-mine car state procedure is as follows:
Monitor video frame sequence, by profound network, completes each convolutional layer and the calculating of extreme value layer, is finally input to full articulamentum.Full interconnect layer data are input in SOFTMAX grader and participate in classified calculating.In SOFTMAX classified calculating process, the data that full articulamentum exports can with the parameter in SOFTMAX grader, carry out the calculating of logistic regression, finally can export about mine car empty, expire the probable value with non-mine car three states, that state of getting maximum probability judges as to the detection of monitor video frame sequence.
Carrying out in the process of above-mentioned judgement to monitoring video frame training, convolutional layer parameter and full articulamentum parameter act as the dimensionality reduction and feature abstraction that realize monitor video frame data, be specially for monitoring image pixel segmentation three passage figure out, at convolutional layer 1, extreme value layer 1, convolutional layer 2, extreme value layer 2, realize the detection of channel image edge and angle point, at convolutional layer 3, maximum layer 3, by combination edge and angle point, realize the detection of channel image shape, at convolutional layer 4 and extreme value layer 4, what complete shape is combined to form the complete description realizing input channel image.And at full articulamentum, by the transmission of receptive field, the Detection Information of each extreme value layer is converged, classify for SOFTMAX.
Each extreme value layer of profound learning network is all connected to full articulamentum, which enhances the information representation of full articulamentum to mine car state under each learning phase, can improve the ability to express of full articulamentum, and then improves Detection results.
In the present invention, the convolution kernel parameter of each convolutional layer, and full articulamentum parameter are all stochastic generation when initial training.By training study, obtain each convolution kernel and coefficient of connection.Meanwhile, the coefficient of full articulamentum, profound network self-teaching, with when following new, is all stochastic generation initial conditions.Such design can avoid over-fitting, obtains higher generalization ability.
Adopt BP algorithm (error backpropagation algorithm) to train profound learning network, iterate, obtain pre-training model.This pre-training model, using the initial model as profound learning network, participates in the judgement of mine car state and the self-teaching of profound learning network, is not therefore very high to the loss of pre-training and required precision.In the training process, we will train the damage control below 10%, and precision controlling is more than 90%.
2, based on profound network, the vehicle-mounted state of mine car is classified
(1) locating candidate region
In monitoring image, full scan mode is adopted to obtain candidate region to be checked.Sweep the average-size arranging the window area by being detected as numeral 0 or 1 according to previous frame of window size, change in the proportion of 0.9 ~ 1.1 of this size.Change window size is out using the size as Current Scan window.It is invalid in a large number to avoid like this, the generation of redundancy scanning window.
(2) profound learning network carries out detection classification
For candidate region, the profound learning network adopting pre-training to practise detects, and obtaining selective mechanisms is true and false candidate region.Here be detected as and really refer to that the digital label of testing result is 0 and 1, namely detect that mine car loading state is empty or is full.The digital label of testing result is 2, represents and is detected as vacation, and namely referring to current is non-mine car cargo area.
3, Sample Refreshment
For the result that profound network detects, be 0,1 to detection digital label result respectively, the candidate region of 2 converts.Concrete transformation rule is: given candidate region, first carries out rotation and tilt variation derives 30 figure, then does gauss change on the basis of 30 figure, derive 30x20 open Gauss's change of scale after figure.Continue on the basis of these figure, add noise, regulate bright and contrast, derive the sample graph of 30x20x10.In addition, be analogous to pre-training process, on the basis of these sample graphs, increase dimensional information.Be specially from present frame, about pushing away 9 frames forward.Based on the 10 frame picture training that this is new, the requirement inputted according to profound network, obtain candidate region in color displacement, the information in shape displacement and in brightness, thus obtain the correlation information of current vehicle-mounted state in time domain and spatial domain.
4, self-teaching and renewal
The sample information of renewal be input in initial profound learning network and finely tune, the weight of the full interconnect layer of profound learning network produces at random, the parameter that profound network others parameter adopts last layer to upgrade.BP algorithm is adopted to train to whole network.Algorithm passes through iterative learning, the new profound learning network model of generation.This model is using as the introductory die model upgraded next time.Adopt above-mentioned way, complete the renewal of profound network learning model.
5, iterate
Monitoring image inputs, and candidate region obtains, and draws current result of determination based on category of model.Sample Refreshment simultaneously, profound network self-teaching, judges for classification next time.Whole process loop iteration.Sample Storehouse constantly increases because of constantly upgrading, and network model has up-to-date expression to vehicle-mounted current scene state simultaneously, and therefore, the learning ability of profound network because of online updating and self-teaching iteration, and constantly can strengthen.
In addition, it is all known for a person skilled in the art that the profound self-teaching real-time performance mine car sky of the embodiment of the present invention expires the detection method of state and other formation of system and effect, in order to reduce redundancy, does not repeat.
In the description of this specification, specific features, structure, material or feature that the description of reference term " embodiment ", " some embodiments ", " example ", " concrete example " or " some examples " etc. means to describe in conjunction with this embodiment or example are contained at least one embodiment of the present invention or example.In this manual, identical embodiment or example are not necessarily referred to the schematic representation of above-mentioned term.And the specific features of description, structure, material or feature can combine in an appropriate manner in any one or more embodiment or example.
Although illustrate and describe embodiments of the invention, those having ordinary skill in the art will appreciate that: can carry out multiple change, amendment, replacement and modification to these embodiments when not departing from principle of the present invention and aim, scope of the present invention is by claim and equivalency thereof.
Claims (10)
1. a detection method for the full state of profound self-teaching real-time performance mine car sky, is characterized in that, comprise the following steps:
A. input multiple mine car dummy status image, the full status image of multiple mine car and multiple non-mine car status image and be stored as initial Sample Storehouse;
B. derivative expansion is carried out to initial Sample Storehouse;
C. build profound learning network to gather monitoring image, according to the described Sample Storehouse after derivative expansion, described monitoring image is analysed and compared, by the described mine car dummy status image stored in described monitoring image and described Sample Storehouse, the full status image of described mine car respectively with described mine car status image comparison difference, according to described comparison difference by described image stored in mine car dummy status image pattern storehouse, the full status image Sample Storehouse of mine car or non-mine car status image Sample Storehouse in described Sample Storehouse; And
D. the described image gathered is carried out derivative expansion in the respective sample storehouse of described Sample Storehouse.
2. the detection method of the full state of profound self-teaching real-time performance mine car sky according to claim 1, it is characterized in that, in stepb, tell the derivative mode expanded mine car dummy status image, the full status image of mine car and the non-mine car status image comprised described Sample Storehouse and carry out affine transformation and/or noise and add and/or brightly to regulate.
3. the detection method of the full state of profound self-teaching real-time performance mine car sky according to claim 1, is characterized in that, in step C, comprise further:
C1. the frame of video choosing N continuous frame is divided into three passages to carry out color displacement, shape displacement and monochrome information respectively extracting, wherein, color displacement refers to carry out difference frame by frame to the RGB passage of video frame pixel described in N frame, and the average of described RGB passage after asking for difference, shape displacement refers to the change in location obtaining the described video frame motion part of monitoring, and monochrome information refers to the gray value directly recording each frame of described frame of video;
C2. the channel information of three described passages carried out convolution and ask for extreme value;
C3. read described mine car dummy status image pattern storehouse, the full status image Sample Storehouse of described mine car and non-mine car status image Sample Storehouse and calculate the difference with described extreme value, by described frame of video stored in the full status image Sample Storehouse of described mine car and the minimum mine car dummy status image pattern storehouse of non-mine car status image Sample Storehouse difference value, the full status image Sample Storehouse of described mine car or non-mine car status image Sample Storehouse.
4. the detection method of the full state of profound self-teaching real-time performance mine car sky according to claim 3, is characterized in that, in step C2, comprise further:
C21. the channel information of three described passages is carried out first time convolution ask for the first extreme value;
C22. carry out second time convolution according to described first extreme value and ask for secondary extremal.
5. the detection method of the full state of profound self-teaching real-time performance mine car sky according to claim 4, is characterized in that, in step C3, comprise further:
C31. described first extreme value and described secondary extremal is obtained;
C32. SOFTMAX is adopted to calculate secondary extremal described in described first extreme value and described secondary extremal and described mine car dummy status image pattern storehouse, the full difference value between status image Sample Storehouse and non-mine car status image Sample Storehouse of described mine car, when described difference value is less than a preset value, by described frame of video stored in corresponding described Sample Storehouse.
6. the detection method of the full state of profound self-teaching real-time performance mine car sky according to claim 1, it is characterized in that, in step D, carried out by the described image gathered derivative expanding in the respective sample storehouse of described Sample Storehouse, the method for described derivative expansion comprises and rotates described image, tilts, introduces noise, regulates at least one item become clear and in adjustment contrast.
7. a detection system for the full state of profound self-teaching real-time performance mine car sky, is characterized in that, comprise
Sample Storehouse, for storing multiple mine car dummy status images of initial input, the full status image of multiple mine car and multiple non-mine car status image;
Derivative enlargement module, for carrying out derivative expansion to described mine car dummy status image, the full status image of described mine car and described non-mine car status image;
Image capture module; And
Select enlargement module, derive the described mine car dummy status image after expansion, the full status image of described mine car and described non-mine car status image and to compare with the image of described image capture module collection respectively for reading described derivative enlargement module and obtain difference value, and according to described difference value by the relevant position of described image stored in described Sample Storehouse.
8. the detection system of the full state of profound self-teaching real-time performance mine car sky according to claim 7, it is characterized in that, described selection enlargement module comprises further:
Read module, derives the described mine car dummy status image after expansion, the full status image of described mine car and described non-mine car status image for reading described derivative enlargement module;
Segmentation extraction module, for being that three passages carry out color displacement respectively by the Iamge Segmentation of video acquisition module collection, the extraction of shape displacement and monochrome information, wherein, color displacement refers to carry out difference frame by frame to the RGB passage of N frame video frame pixel, and the average of described RGB passage after asking for difference, shape displacement refers to the change in location obtaining the described video frame motion part of monitoring, and monochrome information refers to the gray value directly recording each frame of described frame of video;
Convolutional calculation module, carries out convolutional calculation for the channel information that receives from described segmentation extraction module;
Extreme value asks for module, and for asking for extreme value to the result of calculation of convolutional calculation module, described extreme value comprises maximum and minimum;
Sort module, described derivative enlargement module for reading according to described read module derives the described mine car dummy status image after expansion, the full status image of described mine car and described non-mine car status image and asks for ratio of extreme values that module asks for respectively with described extreme value to difference, and classifies to described image according to described difference; And
Enlargement module, for being shown in that sorted described image extends in the relevant position of described Sample Storehouse.
9. the detection system of the full state of profound self-teaching real-time performance mine car sky according to claim 8, it is characterized in that, described convolutional calculation module comprises the first convolution computing module, the second convolution computing module, the 3rd convolutional calculation module and Volume Four and amasss computing module;
Described extreme value is asked for module and is comprised that the first extreme value asks for module, secondary extremal asks for module, the 3rd extreme value asks for module and the 4th extreme value asks for module;
Wherein, described first convolution computing module is for calculating the first volume product value of described channel information, and described first extreme value asks for module for asking for the first extreme value to described first volume product value; Described second convolution computing module is used for calculating volume Two product value according to described first extreme value, and described secondary extremal asks for module for asking for secondary extremal to described volume Two product value; Described 3rd convolutional calculation module is used for calculating the 3rd convolution value according to described secondary extremal, and described 3rd extreme value asks for module for asking for the 3rd extreme value to described 3rd convolution value; Computing module is amassed for calculating Volume Four product value according to described 3rd extreme value in described Volume Four, and described 4th extreme value asks for module for asking for the 4th extreme value to described Volume Four product value.
10. the detection system of the full state of profound self-teaching real-time performance mine car sky according to claim 9, it is characterized in that, described selection enlargement module also comprises full link block, described full link block is for receiving described first extreme value, secondary extremal, the 3rd extreme value and described 4th extreme value, and described first extreme value, secondary extremal, the 3rd extreme value and described 4th extreme value are passed to described sort module classify, described sort module adopts SOFTMAX regression model to classify.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510290352.XA CN104954741B (en) | 2015-05-29 | 2015-05-29 | Profound self-teaching network implementations mine car sky expires the detection method and system of state |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510290352.XA CN104954741B (en) | 2015-05-29 | 2015-05-29 | Profound self-teaching network implementations mine car sky expires the detection method and system of state |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104954741A true CN104954741A (en) | 2015-09-30 |
| CN104954741B CN104954741B (en) | 2019-01-04 |
Family
ID=54169040
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510290352.XA Active CN104954741B (en) | 2015-05-29 | 2015-05-29 | Profound self-teaching network implementations mine car sky expires the detection method and system of state |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104954741B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106815596A (en) * | 2016-12-08 | 2017-06-09 | 中国银联股份有限公司 | A kind of Image Classifier method for building up and device |
| CN106874880A (en) * | 2017-02-21 | 2017-06-20 | 广东大仓机器人科技有限公司 | A kind of method of recognition of face sample-rich |
| CN107316289A (en) * | 2017-06-08 | 2017-11-03 | 华中农业大学 | Crop field spike of rice dividing method based on deep learning and super-pixel segmentation |
| CN108510739A (en) * | 2018-04-28 | 2018-09-07 | 重庆交通大学 | A kind of road traffic state recognition methods, system and storage medium |
| CN108881807A (en) * | 2017-05-09 | 2018-11-23 | 富士通株式会社 | Method and apparatus for being expanded the data in monitor video |
| TWI643137B (en) * | 2017-04-21 | 2018-12-01 | 潘品睿 | Object recognition method and object recognition system |
| CN109086737A (en) * | 2018-08-21 | 2018-12-25 | 武汉恒视途安科技有限公司 | Shipping cargo monitoring video frequency identifying method and system based on convolutional neural networks |
| CN110060265A (en) * | 2019-05-15 | 2019-07-26 | 北京艺泉科技有限公司 | A method of divide from painting and calligraphy cultural relic images and extracts seal |
| CN112462788A (en) * | 2020-12-15 | 2021-03-09 | 济南浪潮高新科技投资发展有限公司 | Balance car automatic following implementation method and system based on mechanical vision and AI technology |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003099778A (en) * | 2001-09-21 | 2003-04-04 | Koto Kado | Image recognition method and device |
| WO2009009697A1 (en) * | 2007-07-11 | 2009-01-15 | Behavioral Recognition Systems, Inc. | Cognitive model for a machine-learning engine in a video analysis system |
| CN104036323A (en) * | 2014-06-26 | 2014-09-10 | 叶茂 | Vehicle detection method based on convolutional neural network |
| CN104463241A (en) * | 2014-10-31 | 2015-03-25 | 北京理工大学 | Vehicle type recognition method in intelligent transportation monitoring system |
-
2015
- 2015-05-29 CN CN201510290352.XA patent/CN104954741B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003099778A (en) * | 2001-09-21 | 2003-04-04 | Koto Kado | Image recognition method and device |
| WO2009009697A1 (en) * | 2007-07-11 | 2009-01-15 | Behavioral Recognition Systems, Inc. | Cognitive model for a machine-learning engine in a video analysis system |
| CN104036323A (en) * | 2014-06-26 | 2014-09-10 | 叶茂 | Vehicle detection method based on convolutional neural network |
| CN104463241A (en) * | 2014-10-31 | 2015-03-25 | 北京理工大学 | Vehicle type recognition method in intelligent transportation monitoring system |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106815596A (en) * | 2016-12-08 | 2017-06-09 | 中国银联股份有限公司 | A kind of Image Classifier method for building up and device |
| CN106874880A (en) * | 2017-02-21 | 2017-06-20 | 广东大仓机器人科技有限公司 | A kind of method of recognition of face sample-rich |
| TWI643137B (en) * | 2017-04-21 | 2018-12-01 | 潘品睿 | Object recognition method and object recognition system |
| CN108881807A (en) * | 2017-05-09 | 2018-11-23 | 富士通株式会社 | Method and apparatus for being expanded the data in monitor video |
| US11037311B2 (en) | 2017-05-09 | 2021-06-15 | Fujitsu Limited | Method and apparatus for augmenting data in monitoring video |
| CN107316289A (en) * | 2017-06-08 | 2017-11-03 | 华中农业大学 | Crop field spike of rice dividing method based on deep learning and super-pixel segmentation |
| CN108510739A (en) * | 2018-04-28 | 2018-09-07 | 重庆交通大学 | A kind of road traffic state recognition methods, system and storage medium |
| CN109086737A (en) * | 2018-08-21 | 2018-12-25 | 武汉恒视途安科技有限公司 | Shipping cargo monitoring video frequency identifying method and system based on convolutional neural networks |
| CN109086737B (en) * | 2018-08-21 | 2021-11-02 | 武汉恒视途安科技有限公司 | Convolutional neural network-based shipping cargo monitoring video identification method and system |
| CN110060265A (en) * | 2019-05-15 | 2019-07-26 | 北京艺泉科技有限公司 | A method of divide from painting and calligraphy cultural relic images and extracts seal |
| CN112462788A (en) * | 2020-12-15 | 2021-03-09 | 济南浪潮高新科技投资发展有限公司 | Balance car automatic following implementation method and system based on mechanical vision and AI technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104954741B (en) | 2019-01-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109902732B (en) | Automatic vehicle classification method and related device | |
| CN104954741A (en) | Tramcar on-load and no-load state detecting method and system based on deep-level self-learning network | |
| CN105590099B (en) | A kind of more people's Activity recognition methods based on improvement convolutional neural networks | |
| CN112052787A (en) | Target detection method and device based on artificial intelligence and electronic equipment | |
| CN105512640A (en) | Method for acquiring people flow on the basis of video sequence | |
| CN109598268A (en) | A kind of RGB-D well-marked target detection method based on single flow depth degree network | |
| CN110852316A (en) | Image tampering detection and positioning method adopting convolution network with dense structure | |
| CN107133974A (en) | The vehicle type classification method that Gaussian Background modeling is combined with Recognition with Recurrent Neural Network | |
| CN105574550A (en) | Vehicle identification method and device | |
| CN107133569A (en) | The many granularity mask methods of monitor video based on extensive Multi-label learning | |
| CN115995063A (en) | Work vehicle detection and tracking method and system | |
| CN104504395A (en) | Method and system for achieving classification of pedestrians and vehicles based on neural network | |
| CN104021381B (en) | Human movement recognition method based on multistage characteristics | |
| CN112241969A (en) | Target detection tracking method and device based on traffic monitoring video and storage medium | |
| CN108734109B (en) | Visual target tracking method and system for image sequence | |
| CN110781882A (en) | License plate positioning and identifying method based on YOLO model | |
| CN112132014B (en) | Target re-identification method and system based on non-supervised pyramid similarity learning | |
| Xing et al. | Traffic sign recognition using guided image filtering | |
| Tourani et al. | A robust vehicle detection approach based on faster R-CNN algorithm | |
| CN114331949A (en) | Image data processing method, computer equipment and readable storage medium | |
| CN110059646A (en) | The method and Target Searching Method of training action plan model | |
| CN113255634A (en) | Vehicle-mounted mobile terminal target detection method based on improved Yolov5 | |
| Xing et al. | The Improved Framework for Traffic Sign Recognition Using Guided Image Filtering | |
| CN116310922A (en) | Petrochemical plant area monitoring video risk identification method, system, electronic equipment and storage medium | |
| Nguyen et al. | Lane detection and tracking based on fully convolutional networks and probabilistic graphical models |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |