CN110929756A - Deep learning-based steel size and quantity identification method, intelligent device and storage medium - Google Patents
Deep learning-based steel size and quantity identification method, intelligent device and storage medium Download PDFInfo
- Publication number
- CN110929756A CN110929756A CN201911013258.4A CN201911013258A CN110929756A CN 110929756 A CN110929756 A CN 110929756A CN 201911013258 A CN201911013258 A CN 201911013258A CN 110929756 A CN110929756 A CN 110929756A
- Authority
- CN
- China
- Prior art keywords
- steel
- size
- image
- face image
- steel pile
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
A steel size and quantity identification method based on deep learning, intelligent equipment and a storage medium are provided. The identification method comprises the following steps: shooting a plurality of steel pile end face images through a camera, acquiring camera aperture values and focal length values corresponding to the steel pile end face images, and training to obtain a steel number detection and size recognition neural network model by utilizing the plurality of steel pile end face images and the aperture values and the focal length values of the camera corresponding to each steel pile end face image; and (3) shooting an end face image of the steel pile to be identified by using a camera, inputting the end face image of the steel pile to be identified and the aperture value and the focal length value of the corresponding camera into the model, and outputting the steel number and size information of the image by the model. So, only need through the terminal surface image of camera collection steel heap and the light ring of camera, focus value, can obtain steel figure, coordinate and the size in the steel heap fast, effectively improved the efficiency and the degree of accuracy of steel count and size discernment, reduce human cost and time cost in a large number.
Description
Technical Field
The invention relates to the technical field of image vision, in particular to a steel size and quantity identification method based on deep learning, intelligent equipment and a storage medium.
Background
In the links of production, transportation, sale and the like of steel products, namely warehousing and selling, for steel enterprises, steel product sellers or steel product buyers, each link needs to accurately calculate the number of the steel products in order to reduce possible economic risks and disputes; meanwhile, the sizes and the thicknesses of various types of steel are different, so that the steel is used differently, and the size of the steel is required to be measured.
At present, the number and size of steel materials are controlled by the following methods:
1. and (5) manually counting and measuring.
Manual marking and measurement are the most common methods at the present stage, and workers mark steel products one by using pens with different colors. The traditional counting mode is time-consuming and labor-consuming, and has low speed, low accuracy and potential safety hazard. Further, although the building-related materials such as steel materials, round steel, pipes, beads, etc. and other materials are marked with information such as specifications and quantities including information on the place of production, it is difficult to grasp the exact quantities of all the materials because of the different specifications and the irregular arrangement or stacking in the factory, and particularly, the steel products are difficult to move because of their heavy weights, which makes counting difficult.
2. The automatic counting is carried out by using a photoelectric tube and a pulse sensor. The method has the following defects: 1) only single measurement can be carried out, so that the efficiency is low; 2) under severe environment, the sensitivity of the photoelectric tube is reduced, and the error is larger; 3) the dimensions of the steel cannot be measured simultaneously.
3. A weight sensor detection method. The method is too dependent on the steel production process, and under the condition that the steel factory has negative tolerance production, the error of the number of the steel measured by the method is large, and the size of the steel cannot be measured simultaneously.
4. A computer vision method. At the present stage, a few of researchers in the image field use a traditional algorithm or an artificial intelligence method to count the steel products, and the method can output the quantity of the steel products only by inputting the steel product map into a designed algorithm model, thereby liberating the manpower, greatly improving the efficiency and reducing the cost. However, these methods at the present stage have the following problems: 1) for the traditional algorithm, the actual storage environment of the steel is complex and changeable, and the situations of light change, steel shielding and the like are easy to occur, the traditional algorithm has high requirements on a scene, and the practicality of the traditional algorithm is not very strong due to the variability of the scene; 2) for some deep learning algorithms at the present stage, although the model has a high theoretical detection rate due to the diversity of scenes and the incompleteness of data, the accuracy and recall rate of steel number detection in actual use are not high. Meanwhile, the diameter variation range of steel is large (various types between 12 and 32), the interface is irregular in shape and different in color, and the shooting angle and distance are not controlled, so that the detection effect of the algorithm model is difficult to stabilize in the actual use process; 3) in the existing method, a method which can realize quantity statistics of steel and can classify the size grade of the steel bundle does not exist at the same time.
Aiming at the problems, the invention provides an intelligent detection method capable of simultaneously detecting the quantity and the size of steel products based on deep learning and a computer vision theory. Under the condition that steel storage environments such as construction sites are complex and changeable, the method can realize the functions of steel counting and size identification at the same time, and aims to detect the quantity and identify the size of the steel more accurately, efficiently and quickly, reduce manpower and improve production efficiency.
Disclosure of Invention
The invention aims to provide a steel size and quantity identification method based on deep learning aiming at the problems in the prior art, and the method helps us to carry out quick, efficient and accurate quantity and size statistics on batch steel.
Another object of the present invention is to provide a smart device.
It is yet another object of the present invention to provide a computer storage medium.
In order to realize the purpose of the invention, the adopted technical scheme is as follows: a steel size and quantity identification method based on deep learning comprises the following steps:
step S1, shooting a plurality of steel pile end face images through a camera, and training a steel number detection and size recognition neural network model by using the plurality of steel pile end face images and the aperture value and the focal length value of the camera corresponding to each steel pile end face image;
and step S2, shooting an end face image of the steel pile to be identified by using a camera, and inputting the end face image of the steel pile to be identified and an aperture value and a focal length value of the corresponding camera into the steel number detection and size identification neural network model, wherein the steel number detection and size identification neural network model is used for obtaining the steel number, the position of each steel and the size information of each steel in the end face image of the steel pile to be identified.
By adopting the identification method, the number of the steel materials in the steel material pile, the coordinates of the steel materials and the size of the steel materials can be obtained only by acquiring the end face image of the steel material pile and the aperture and the focal length value of the current camera. Therefore, the efficiency and the accuracy of steel counting and size identification are effectively improved, and the labor cost and the time cost are greatly reduced.
As the optimization of the method for identifying the size and the number of steel products based on deep learning according to the present invention, the step S1 includes:
step S11, collecting training data samples, wherein the training data samples include: shooting a steel pile by a camera under different shooting angles, weather and illumination conditions to obtain an end face image and a background image of the steel pile; the aperture value and the focal length value of the camera during each shooting;
step S13, step S13, carrying out normalization mapping processing on each training data sample, and mapping the pixel value, the aperture value and the focal length value of the steel pile end face image into an interval [0, 1 ];
and step S14, manually marking the steel pile end face images, namely marking the number of steel products, the position coordinates of each steel product and the size of each steel product in each steel pile end face image, and taking the marked steel products as the training basis of the steel product number detection and size identification neural network model.
As the optimization of the method for identifying the size and the number of steel products based on deep learning provided by the present invention, the step S1 further includes:
and step S12, performing data set expansion on the training data sample by one or more modes of basic image transformation, steel dynamic random Dropout method and dynamic background difficulty sample enhancement method.
As the optimization of the method for identifying the size and the number of steel products based on deep learning provided by the present invention, the basic image transformation includes: and carrying out one or more operations of image blurring, image channel pixel transformation, image random noise elimination, image affine transformation and slight image rotation on the collected steel pile end face image to obtain a first-class expansion diagram, and adding the first-class expansion diagram obtained after the operation and an aperture value and a focal length value corresponding to the corresponding steel pile end face image into the training data sample to carry out basic expansion of a data set.
As the optimization of the method for identifying the size and the number of the steel products based on the deep learning provided by the invention, the steel product dynamic random Dropout method comprises the following steps: and randomly selecting a part of steel from one pair of steel pile end face images to carry out pixel zero erasing operation, namely discarding the part of steel to obtain a second type of expansion diagram, and adding the second type of expansion diagram and the aperture value and the focal length value corresponding to the corresponding steel pile end face image into the training data sample to carry out secondary expansion of the data set.
As the optimization of the identification method of the size and the quantity of the steel based on deep learning, the dynamic background difficult sample enhancement method comprises the following steps: collecting a plurality of construction site background images with different environments, cutting a steel pile in a pair of steel pile end face images, pasting the cut steel pile in the construction site background images randomly to obtain a third type of expansion image, and adding the third type of expansion image and an aperture value and a focal length value corresponding to the corresponding steel pile end face image into the training data sample to perform three times of expansion of a data set.
In the optimization of the method for identifying the size and the number of steel products based on the deep learning according to the present invention, in step S14, the sizes of the steel products are classified into classes according to the specification and the size of the steel products, and when the sizes of the steel products are labeled, the corresponding steel products are labeled with the class of the size.
As the optimization of the method for identifying the size and the number of steel products based on deep learning according to the present invention, the step S2 includes:
step S21, obtaining a feature atlas of the end face image of the steel pile to be identified through a basic feature extraction network;
step S22, extracting a target candidate frame of the steel in the end face image of the steel pile to be identified through an RPN;
step S23, combining the feature atlas and the target candidate frame information to perform ROI pooling operation to obtain a candidate feature atlas;
and S24, inputting corresponding aperture values and focal length values as size input feature items, combining the candidate feature atlas obtained in the step S23, stacking and inputting the candidate feature atlas into a full-connection feature classification regression network, and obtaining the classification category and confidence coefficient of each steel material, the position coordinate information of each steel material and the size level confidence coefficient of each steel material in the end face image of the steel material stack to be identified.
S25, screening score thresholds of the results of the step S24 to obtain target candidate frames of each steel with the confidence coefficient above the threshold, and carrying out number statistics and coordinate statistics on the obtained target candidate frames to obtain the number of the steel and the position coordinates of each steel; and sequencing the confidence of the size level of each steel, and selecting the size level with the maximum confidence value as the size result of the corresponding steel.
To achieve another object of the present invention, the present invention also provides a smart device, characterized in that the smart device comprises a processor and a memory; the memory stores a computer program; the processor is used for running the computer program to realize the identification method of the steel size and quantity based on deep learning.
To achieve still another object of the present invention, there is also provided a computer storage medium storing a computer program which, when executed, implements the deep learning-based steel product size and quantity identification method as described above.
The steel automatic identification and counting method based on deep learning provided by the invention can achieve the following beneficial effects: the steel size and quantity identification method based on deep learning comprises the following steps: step S1, a plurality of steel pile end face images are shot by a camera, and a steel number detection and size recognition neural network model is trained by the aid of the plurality of steel pile end face images and aperture values and focal length values of the camera corresponding to each steel pile end face image; and step S2, shooting an end face image of the steel pile to be identified by using a camera, and inputting the end face image of the steel pile to be identified and an aperture value and a focal length value of the corresponding camera into the steel number detection and size identification neural network model, wherein the steel number detection and size identification neural network model is used for obtaining the steel number and size information in the end face image of the steel pile to be identified. By adopting the identification method, the number of the steel materials in the steel material pile, the coordinates of the steel materials and the size of the steel materials can be obtained only by acquiring the end face image of the steel material pile and the aperture and the focal length value of the current camera. Therefore, the efficiency and the accuracy of steel counting and size identification are effectively improved, and the labor cost and the time cost are greatly reduced.
Drawings
FIG. 1 is a flowchart of step S1 according to one embodiment of the present invention;
FIG. 2 is a photograph of an image of an end face of a steel material according to a first embodiment of the present invention;
fig. 3 is a flowchart of step S2 according to an embodiment of the present invention.
Detailed Description
For a more clear understanding of the technical features, objects and effects of the present invention, embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example one
The method for identifying the size and the number of the steel products based on deep learning provided by the embodiment comprises the following steps:
step S1, a plurality of steel pile end face images are shot by a camera, and a steel number detection and size recognition neural network model is trained by the aid of the plurality of steel pile end face images and aperture values and focal length values of the camera corresponding to each steel pile end face image;
it should be noted that the steel material mentioned in the present invention is a long strip steel material, and of course, in other embodiments, the steel material may be round steel, steel pipe, steel ball, or other building-related materials. The end face image of the steel pile can be obtained by photographing the end face of the steel pile to be checked through a smart phone or other equipment with a camera shooting function.
In this embodiment, the step S1 includes:
step S11, collecting training data samples, wherein the training data samples include: shooting a steel pile by a camera under different shooting angles, weather and illumination conditions to obtain an end face image and a background image of the steel pile; the aperture value and the focal length value of the camera during each shooting;
in this embodiment, in order to calculate the size of the steel material, the present invention needs to collect the aperture and focal length information (i.e., the aperture value and the focal length value) of the camera when shooting each image. The invention considers the pictures under most steel counting and size identification application scenes, including different shooting angles, illumination, shielding, steel bundle density and dispersion and other conditions, thereby improving the accuracy of the algorithm model in the steel number statistics and size identification in practical application.
Step S12, performing data set expansion on the training data sample by one or more modes of basic image transformation, steel dynamic random Dropout method and dynamic background difficult sample enhancement method;
specifically, the basic image transformation includes: and carrying out one or more operations of image blurring, image channel pixel transformation, image random noise elimination, image affine transformation and slight image rotation on the collected steel pile end face image to obtain a first type of expansion diagram, and adding the first type of expansion diagram obtained after the operation and an aperture value and a focal length value corresponding to the corresponding steel pile end face image into the training data sample to expand the data set. Assuming that the acquired end face image of the steel pile is I (x, y), the first type of expansion map generated by the transformation is I' (x, y), and the above image enhancement transformation can be summarized in the following mathematical model:
I′(x,y)＝T(x,y,θ,L)I(x,y)，
where T (x, y, θ, L) is a transformation matrix, and represents a transformed image obtained based on different angles (which may be a refraction angle or a rotation angle).
The dynamic Dropout method for the steel materials comprises the steps of randomly selecting a part of steel materials in a pair of steel material pile end face images to carry out pixel zero-erasing operation, namely discarding the part of the steel materials to obtain a second type of expansion map, and adding the second type of expansion map and aperture values and focal length values corresponding to the corresponding steel material pile end face images into a training data sample to expand a data set, so that the diversity of the sample is increased, the overfitting of a model is reduced, and the generalization capability of the model is enhanced.
Specifically, the dynamic background difficult sample enhancement method includes: collecting a plurality of construction site background images with different environments, cutting a steel pile in a pair of steel pile end face images, pasting the cut steel pile in the construction site background images randomly to obtain a third type of expansion diagram, and adding the third type of expansion diagram and aperture values and focal length values corresponding to the corresponding steel pile end face images into the training data sample to expand the data set. In addition, a dynamic background difficult sample set can be generated after the steel number detection and size recognition neural network model is subjected to a certain training epochs, the dynamic background difficult sample set is adopted to train the steel number detection and size recognition model again, and the steel number detection and size recognition model is optimized and purified so as to perfect the steel number detection and size recognition capability of the steel number detection and size recognition model in a complex and variable scene.
Step S13, carrying out normalization mapping processing on each training data sample, and mapping the value of each training data sample to an interval [0, 1 ];
specifically, the data range of the steel number detection and size identification neural network model is mostly 0-1. In order to detect the number of steel products and learn and converge a size identification model, normalization operation of 0-1 is conducted on input data. For example, the pixel range of the steel stack end face image is 0 to 255, and the normalization processing of the steel stack end face image is performed by using a mean variance normalization method. Assuming that a certain pixel of a certain color channel of the end face image of the steel pile has the intensity x, the normalization formula is as follows:
wherein x' is the normalized pixel intensity,
is the mean of the pixel intensities of the training sample set, and σ is the standard deviation of the pixel intensities of the training sample set.
In addition, data mapping of 0 to 1 is also required for the focal length value and the aperture value. The mapping of the focal length value can be linear equilibrium mapping or nonlinear mapping of numerical values. For discrete data of aperture value, a discrete mapping method is adopted, and the reference mapping table is as follows.
| Aperture | F1 | F1.4 | F2 | F2.8 | F4 | F5.6 | F8 |
| Mapping values | 0.05 | 0.1 | 0.15 | 0.2 | 0.25 | 0.3 | 0.35 |
| Aperture | F11 | F16 | F22 | F32 | F44 | F64 | … |
| Mapping values | 0.4 | 0.45 | 0.5 | 0.55 | 0.6 | 0.65 | … |
And step S14, manually marking the steel pile end face images, namely marking the number of steel products, the position coordinates of each steel product and the size of each steel product in each steel pile end face image, and taking the marked number of steel products, position coordinates of each steel product and size of each steel product as a training basis of a steel product number detection and size identification model.
It is to be noted that, in the step S14, the dimensions of the steel material are classified according to the specification and size of the steel material, and when the dimensions of the steel material are marked, the corresponding steel material is marked with the grade of the dimensions. For example, for the size of the steel material, the size of the steel material can be divided into 8 size grades (10, 12, 14, 16, 18, 20, 22, 25), and when the size of the steel material is marked, only the numbers 10, 12, 14, 16, 18, 20, 22, 25 need to be marked, so that the task of identifying the size of the steel material is converted into a simpler task of classifying, the difficulty of identifying the size of the steel material is reduced, the calculation rate is reduced, and the speed of identifying the size of the steel material is increased.
And step S2, shooting an end face image of the steel pile to be identified by using a camera, and inputting the end face image of the steel pile to be identified and an aperture value and a focal length value of the corresponding camera into the steel number detection and size identification neural network model, wherein the steel number detection and size identification neural network model is used for obtaining the steel number, coordinates and size information in the end face image of the steel pile to be identified.
In this embodiment, the step S2 includes:
step S21, feature extraction: obtaining a characteristic atlas of the end face image of the steel pile to be identified through a basic characteristic extraction network; specifically, the end face image of the steel pile to be identified is input into a basic feature extraction network, and a feature atlas of the end face image of the steel pile to be identified is obtained through a series of convolution, pooling and batch normalization operations.
Step S22, target candidate box extraction: extracting a target candidate frame of the steel in the end face image of the steel pile to be identified through an RPN; here, based on the frame regression theory, a steel target candidate region is screened using a region candidate network (RPN) to obtain a plurality of steel target candidate frames. The method specifically comprises the following steps:
1) generating anchor anchors: vector T transformation using bounding box regressionbbox(A) Performing border regression on all anchor points (the anchor points are completely consistent during generation and training), wherein a transformation vector of the border regression is:
Tbbox(A)＝[dx(A),dy(A),dw(A),dh(A)]；
2) the Softmax classifier extracts positive anchor points (positvie anchors): according to the input positive score of softmax, the anchors are sorted from big to small, and the first N anchors are extracted, namely the positive anchor anchors _ P after the correction position is extracted;
3) adopting a non-maximum inhibition method to regress and screen out positive anchor points: defining anchors _ P beyond the image boundary as the image boundary to prevent propofol from exceeding the image boundary upon subsequent ROI pooling; meanwhile, eliminating the positive anchor points anchors _ P which are very small (namely the width and the height of the frame are smaller than the threshold value), and performing frame fusion by using a non-maximum inhibition method;
4) the candidate Layer (Proposal Layer) generates a target candidate frame.
Step S23, ROI pooling: combining the feature atlas and the target candidate frame information to perform ROI pooling operation to obtain a candidate feature atlas;
step S24, feature classification regression: and (4) inputting corresponding aperture values and focal length values as size input feature items, combining the candidate feature atlas obtained in the step (23), stacking and inputting the candidate feature atlas into a full-connection feature classification regression network, and obtaining the classification category and confidence coefficient of each steel material, the position coordinate information of each steel material and the size level confidence coefficient of each steel material in the end face image of the steel material stack to be identified.
Step S25, post-processing: screening score thresholds of the results of the step S24 to obtain target candidate frames of the steel products with confidence degrees above the thresholds, and performing number statistics and coordinate statistics on the obtained target candidate frames to obtain the number of the steel products and the position coordinates of each steel product; and sequencing the confidence of the size level of each steel, and selecting the size level with the maximum confidence value as the size result of the corresponding steel.
In summary, the steel automatic identification and counting method based on deep learning provided by the invention can achieve the following beneficial effects:
1. the number of the steel materials in the steel material pile, the coordinates of the steel materials and the size of the steel materials can be obtained through the model only by acquiring the end face image of the steel material pile and the aperture and the focal length value of the current camera. Therefore, the efficiency and the accuracy of steel counting and size identification are effectively improved, and the labor cost and the time cost are greatly reduced.
2. In step S13, the normalized mapping operation is performed on the aperture value and the focal length value, so that the steel number detection and size identification model can be effectively converged, and the identification and classification of the steel size can be accurately realized.
3. The training data samples are enlarged through a dynamic background difficult sample enhancement method, so that the training data samples contain a larger number of samples with different construction site environments, different steel shapes and different shooting degrees, and the accuracy of counting and size identification of the steel number detection and size identification neural network model in an actual application scene is improved.
5. The training data samples are expanded through a steel dynamic random Dropout method, so that the training data samples comprise samples in different forms and different numbers, the samples are expanded, and the generalization capability of a steel number detection and size identification model is enhanced.
Example two
The embodiment provides an intelligent device, which comprises a processor and a memory; the memory stores a computer program; the processor is used for running the computer program to realize the deep learning-based steel size and quantity identification method provided by the embodiment one. The intelligent device can be a smart phone or a tablet computer with a camera shooting function. Of course, a computer signal-connected to the image pickup apparatus may be used.
EXAMPLE III
A computer storage medium storing a computer program; the computer program is executed to implement the deep learning-based steel size and quantity identification method provided in the first embodiment. Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be implemented by a program instructing associated hardware, the program may be stored in a computer storage medium including Read-Only Memory (ROM), Random Access Memory (RAM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), One-time Programmable Read-Only Memory (OTPROM), Electrically Erasable rewritable Read-Only Memory (EEPROM), compact disc Read-Only Memory (CD-ROM) or other optical disc storage, magnetic disk storage, tape storage, or any other medium readable by a computer that can be used to carry or store data.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (10)
1. A steel size and quantity identification method based on deep learning is characterized by comprising the following steps:
step S1, shooting a plurality of steel pile end face images through a camera, and training a steel number detection and size recognition neural network model by using the plurality of steel pile end face images and the aperture value and the focal length value of the camera corresponding to each steel pile end face image;
and step S2, shooting an end face image of the steel pile to be identified by using a camera, and inputting the end face image of the steel pile to be identified and an aperture value and a focal length value of the corresponding camera into the steel number detection and size identification neural network model, wherein the steel number detection and size identification neural network model is used for obtaining the steel number, the position of each steel and the size information of each steel in the end face image of the steel pile to be identified.
2. The steel product size and quantity recognition method based on deep learning of claim 1, wherein the step S1 includes:
step S11, collecting training data samples, wherein the training data samples include: shooting a steel pile by a camera under different shooting angles, weather and illumination conditions to obtain an end face image and a background image of the steel pile; the aperture value and the focal length value when the camera shoots each image;
step S13, performing normalization mapping processing on each training data sample, and mapping the pixel value, the aperture value and the focal length value of the steel pile end face image into an interval [0, 1 ];
and step S14, manually marking the steel pile end face images, namely marking the number of steel products, the position coordinates of each steel product and the size of each steel product in each steel pile end face image, and taking the marked steel products as the training basis of the steel product number detection and size identification neural network model.
3. The steel product size and quantity recognition method based on deep learning of claim 2, wherein the step S1 further includes:
and step S12, performing data set expansion on the training data sample by one or more modes of basic image transformation, steel dynamic random Dropout method and dynamic background difficulty sample enhancement method.
4. The method for identifying the size and the number of steel products based on deep learning according to claim 3, wherein the basic image transformation comprises: and carrying out one or more operations of image blurring, image channel pixel transformation, image random noise elimination, image affine transformation and slight image rotation on the collected steel pile end face image to obtain a first-class expansion diagram, and adding the first-class expansion diagram obtained after the operation and an aperture value and a focal length value corresponding to the corresponding steel pile end face image into the training data sample to carry out basic expansion of a data set.
5. The steel product size and quantity identification method based on deep learning of claim 3, wherein the steel product dynamic random Dropout method comprises: and randomly selecting a part of steel from the steel pile end face image to carry out pixel zero-erasing operation, namely discarding the part of steel to obtain a second type of expansion diagram, and adding the second type of expansion diagram and an aperture value and a focal length value corresponding to the corresponding steel pile end face image into the training data sample to carry out secondary expansion of the data set.
6. The steel product size and quantity identification method based on deep learning of claim 3, wherein the dynamic background difficulty sample enhancement method comprises: collecting a plurality of construction site background images with different environments, cutting a steel pile in a pair of steel pile end face images, pasting the cut steel pile in the construction site background images randomly to obtain a third type of expansion diagram, and adding the third type of expansion diagram and aperture values and focal length values corresponding to the corresponding steel pile end face images into the training data sample to perform three times of expansion of a data set.
7. The method for identifying the size and quantity of steel products based on deep learning as claimed in claim 2, wherein in step S14, the sizes of the steel products are classified into classes according to the specification and size of the steel products, and when the sizes of the steel products are labeled, the corresponding steel products are labeled with the class of the sizes.
8. The steel product size and quantity recognition method based on deep learning of claim 1, wherein the step S2 includes:
step S21, obtaining a feature atlas of the end face image of the steel pile to be identified through a basic feature extraction network;
step S22, extracting a target candidate frame of the steel in the end face image of the steel pile to be identified through an RPN;
step S23, combining the feature atlas and the target candidate frame information to perform ROI pooling operation to obtain a candidate feature atlas;
and step S24, inputting corresponding aperture values and focal length values as size input feature items, combining the candidate feature atlas obtained in the step 23, stacking and inputting the candidate feature atlas into a full-connection feature classification regression network, and obtaining the classification category and confidence coefficient of each steel material, the position coordinate information of each steel material and the size level confidence coefficient of each steel material in the end face image of the steel material pile to be recognized.
S25, screening score thresholds of the results of the step S24 to obtain target candidate frames of each steel with the confidence coefficient above the threshold, and carrying out number statistics and coordinate statistics on the obtained target candidate frames to obtain the number of the steel and the position coordinates of each steel; and sequencing the confidence of the size level of each steel, and selecting the size level with the maximum confidence value as the size result of the corresponding steel.
9. A smart device, wherein the smart device comprises a processor and a memory; the memory stores a computer program; the processor is used for running the computer program to realize the identification method of the steel size and quantity based on the deep learning of any one of claims 1 to 8.
10. A computer storage medium, characterized by a computer program stored therein; the computer program is executed to implement the deep learning-based steel product size and quantity identification method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911013258.4A CN110929756B (en) | 2019-10-23 | 2019-10-23 | Steel size and quantity identification method based on deep learning, intelligent equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911013258.4A CN110929756B (en) | 2019-10-23 | 2019-10-23 | Steel size and quantity identification method based on deep learning, intelligent equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110929756A true CN110929756A (en) | 2020-03-27 |
| CN110929756B CN110929756B (en) | 2022-09-06 |
Family
ID=69849272
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911013258.4A Active CN110929756B (en) | 2019-10-23 | 2019-10-23 | Steel size and quantity identification method based on deep learning, intelligent equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110929756B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111523429A (en) * | 2020-04-16 | 2020-08-11 | 中冶赛迪重庆信息技术有限公司 | Deep learning-based steel pile identification method |
| CN111784281A (en) * | 2020-06-10 | 2020-10-16 | 中国铁塔股份有限公司 | Asset identification method and system based on AI |
| CN111968104A (en) * | 2020-08-27 | 2020-11-20 | 中冶赛迪重庆信息技术有限公司 | Machine vision-based steel coil abnormity identification method, system, equipment and medium |
| CN112037196A (en) * | 2020-08-31 | 2020-12-04 | 中冶赛迪重庆信息技术有限公司 | Cooling bed multiple-length detection method, system and medium |
| CN112037195A (en) * | 2020-08-31 | 2020-12-04 | 中冶赛迪重庆信息技术有限公司 | Method, system, equipment and medium for detecting abnormal length of bar |
| CN112053335A (en) * | 2020-08-31 | 2020-12-08 | 中冶赛迪重庆信息技术有限公司 | Hot-rolled bar overlapping detection method, system and medium |
| CN112053337A (en) * | 2020-08-31 | 2020-12-08 | 中冶赛迪重庆信息技术有限公司 | Bar detection method, device and equipment based on deep learning |
| CN112053333A (en) * | 2020-08-31 | 2020-12-08 | 中冶赛迪重庆信息技术有限公司 | Square billet detection method, system, equipment and medium based on machine vision |
| CN112800887A (en) * | 2021-01-16 | 2021-05-14 | 中建三局第一建设工程有限责任公司 | Columnar target object recognition system |
| CN113674200A (en) * | 2021-07-08 | 2021-11-19 | 浙江大华技术股份有限公司 | Method and device for counting articles on production line and computer storage medium |
| CN114783144A (en) * | 2022-06-17 | 2022-07-22 | 深圳市易智博网络科技有限公司 | Intelligent building site safety monitoring method and device and computer equipment |
| CN116245962A (en) * | 2023-03-16 | 2023-06-09 | 祝晓鹏 | Data extraction system and method for wireless transmission to blockchain server |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPS618608A (en) * | 1984-06-22 | 1986-01-16 | Kawasaki Steel Corp | Method for measuring size of u-shaped steel sheet pile |
| JPH06117838A (en) * | 1992-10-01 | 1994-04-28 | Nippon Steel Corp | Image inputting device |
| JP2006031440A (en) * | 2004-07-16 | 2006-02-02 | Konica Minolta Photo Imaging Inc | Image processing method, image processing apparatus, image processing program and image processing system |
| CN1905629A (en) * | 2005-07-26 | 2007-01-31 | 佳能株式会社 | Image capturing apparatus and image capturing method |
| CN101545774A (en) * | 2009-04-30 | 2009-09-30 | 上海交通大学 | Method for measuring distance based on single image |
| CN102999451A (en) * | 2012-11-13 | 2013-03-27 | 上海交通大学 | Steel counting system and steel counting method |
| CN103353282A (en) * | 2013-06-09 | 2013-10-16 | 河北工业大学 | On-line vision measurement apparatus for end shapes of hot rolled strip steel |
| CN106651870A (en) * | 2016-11-17 | 2017-05-10 | 山东大学 | Method for segmenting out-of-focus fuzzy regions of images in multi-view three-dimensional reconstruction |
| CN107330390A (en) * | 2017-06-26 | 2017-11-07 | 上海远洲核信软件科技股份有限公司 | A kind of demographic method based on graphical analysis and deep learning |
| CN108230292A (en) * | 2017-04-11 | 2018-06-29 | 北京市商汤科技开发有限公司 | The training method of object detecting method and neural network, device and electronic equipment |
| CN109064511A (en) * | 2018-08-22 | 2018-12-21 | 广东工业大学 | A kind of gravity center of human body's height measurement method, device and relevant device |
| CN109977876A (en) * | 2019-03-28 | 2019-07-05 | 腾讯科技（深圳）有限公司 | Image-recognizing method, calculates equipment, system and storage medium at device |
| CN109993094A (en) * | 2019-03-26 | 2019-07-09 | 苏州富莱智能科技有限公司 | Fault in material intelligent checking system and method based on machine vision |
| CN110032954A (en) * | 2019-03-27 | 2019-07-19 | 成都数之联科技有限公司 | A kind of reinforcing bar intelligent recognition and method of counting and system |
| CN110349201A (en) * | 2019-07-07 | 2019-10-18 | 创新奇智(合肥)科技有限公司 | A kind of suit length measurement method, system and electronic equipment neural network based |
-
2019
- 2019-10-23 CN CN201911013258.4A patent/CN110929756B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPS618608A (en) * | 1984-06-22 | 1986-01-16 | Kawasaki Steel Corp | Method for measuring size of u-shaped steel sheet pile |
| JPH06117838A (en) * | 1992-10-01 | 1994-04-28 | Nippon Steel Corp | Image inputting device |
| JP2006031440A (en) * | 2004-07-16 | 2006-02-02 | Konica Minolta Photo Imaging Inc | Image processing method, image processing apparatus, image processing program and image processing system |
| CN1905629A (en) * | 2005-07-26 | 2007-01-31 | 佳能株式会社 | Image capturing apparatus and image capturing method |
| CN101545774A (en) * | 2009-04-30 | 2009-09-30 | 上海交通大学 | Method for measuring distance based on single image |
| CN102999451A (en) * | 2012-11-13 | 2013-03-27 | 上海交通大学 | Steel counting system and steel counting method |
| CN103353282A (en) * | 2013-06-09 | 2013-10-16 | 河北工业大学 | On-line vision measurement apparatus for end shapes of hot rolled strip steel |
| CN106651870A (en) * | 2016-11-17 | 2017-05-10 | 山东大学 | Method for segmenting out-of-focus fuzzy regions of images in multi-view three-dimensional reconstruction |
| CN108230292A (en) * | 2017-04-11 | 2018-06-29 | 北京市商汤科技开发有限公司 | The training method of object detecting method and neural network, device and electronic equipment |
| CN107330390A (en) * | 2017-06-26 | 2017-11-07 | 上海远洲核信软件科技股份有限公司 | A kind of demographic method based on graphical analysis and deep learning |
| CN109064511A (en) * | 2018-08-22 | 2018-12-21 | 广东工业大学 | A kind of gravity center of human body's height measurement method, device and relevant device |
| CN109993094A (en) * | 2019-03-26 | 2019-07-09 | 苏州富莱智能科技有限公司 | Fault in material intelligent checking system and method based on machine vision |
| CN110032954A (en) * | 2019-03-27 | 2019-07-19 | 成都数之联科技有限公司 | A kind of reinforcing bar intelligent recognition and method of counting and system |
| CN109977876A (en) * | 2019-03-28 | 2019-07-05 | 腾讯科技（深圳）有限公司 | Image-recognizing method, calculates equipment, system and storage medium at device |
| CN110349201A (en) * | 2019-07-07 | 2019-10-18 | 创新奇智(合肥)科技有限公司 | A kind of suit length measurement method, system and electronic equipment neural network based |
Non-Patent Citations (6)
| Title |
|---|
| DEEPAK BABU SAM等: "Locate, Size and Count: Accurately Resolving People in Dense Crowds via Detection", 《ARXIV:1906.07538V2》 * |
| EDWARD LAURENCE等: "CAMERA SETTINGS PREDICTION USING DEEP CONVOLUTIONAL", 《EDWARDLAURENCE》 * |
| 刘崇展: "智能化粮库粮面高度测量研究及应用", 《中国优秀硕士学位论文全文数据库 农业科技辑》 * |
| 陈爱弟等: "加工过程中尺寸在线检测技术", 《制造业自动化》 * |
| 马振涛: "柱状喷射沉积坯动态特征尺寸检测与神经网络仿真的研究", 《中国优秀硕士学位论文全文数据库 工程科技I辑》 * |
| 高金刚等: "双目立体视觉在动车车身关键尺寸检测中的应用", 《计算机应用》 * |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111523429A (en) * | 2020-04-16 | 2020-08-11 | 中冶赛迪重庆信息技术有限公司 | Deep learning-based steel pile identification method |
| CN111784281A (en) * | 2020-06-10 | 2020-10-16 | 中国铁塔股份有限公司 | Asset identification method and system based on AI |
| CN111968104A (en) * | 2020-08-27 | 2020-11-20 | 中冶赛迪重庆信息技术有限公司 | Machine vision-based steel coil abnormity identification method, system, equipment and medium |
| CN111968104B (en) * | 2020-08-27 | 2023-04-07 | 中冶赛迪信息技术(重庆)有限公司 | Machine vision-based steel coil abnormity identification method, system, equipment and medium |
| CN112053333B (en) * | 2020-08-31 | 2023-04-07 | 中冶赛迪信息技术(重庆)有限公司 | Square billet detection method, system, equipment and medium based on machine vision |
| CN112053335A (en) * | 2020-08-31 | 2020-12-08 | 中冶赛迪重庆信息技术有限公司 | Hot-rolled bar overlapping detection method, system and medium |
| CN112053337A (en) * | 2020-08-31 | 2020-12-08 | 中冶赛迪重庆信息技术有限公司 | Bar detection method, device and equipment based on deep learning |
| CN112053333A (en) * | 2020-08-31 | 2020-12-08 | 中冶赛迪重庆信息技术有限公司 | Square billet detection method, system, equipment and medium based on machine vision |
| CN112037195A (en) * | 2020-08-31 | 2020-12-04 | 中冶赛迪重庆信息技术有限公司 | Method, system, equipment and medium for detecting abnormal length of bar |
| CN112037195B (en) * | 2020-08-31 | 2023-04-07 | 中冶赛迪信息技术(重庆)有限公司 | Method, system, equipment and medium for detecting abnormal length of bar |
| CN112037196A (en) * | 2020-08-31 | 2020-12-04 | 中冶赛迪重庆信息技术有限公司 | Cooling bed multiple-length detection method, system and medium |
| CN112800887A (en) * | 2021-01-16 | 2021-05-14 | 中建三局第一建设工程有限责任公司 | Columnar target object recognition system |
| CN112800887B (en) * | 2021-01-16 | 2022-06-07 | 中建三局第一建设工程有限责任公司 | Columnar target recognition system |
| CN113674200A (en) * | 2021-07-08 | 2021-11-19 | 浙江大华技术股份有限公司 | Method and device for counting articles on production line and computer storage medium |
| CN114783144A (en) * | 2022-06-17 | 2022-07-22 | 深圳市易智博网络科技有限公司 | Intelligent building site safety monitoring method and device and computer equipment |
| CN116245962A (en) * | 2023-03-16 | 2023-06-09 | 祝晓鹏 | Data extraction system and method for wireless transmission to blockchain server |
| CN116245962B (en) * | 2023-03-16 | 2023-12-22 | 新疆量子通信技术有限公司 | Data extraction system and method for wireless transmission to blockchain server |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110929756B (en) | 2022-09-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110929756B (en) | Steel size and quantity identification method based on deep learning, intelligent equipment and storage medium | |
| CN108830188B (en) | Vehicle detection method based on deep learning | |
| Beery et al. | Recognition in terra incognita | |
| CN108510467B (en) | SAR image target identification method based on depth deformable convolution neural network | |
| CN113065558A (en) | Lightweight small target detection method combined with attention mechanism | |
| US8340420B2 (en) | Method for recognizing objects in images | |
| CN108038846A (en) | Transmission line equipment image defect detection method and system based on multilayer convolutional neural networks | |
| CN111695514B (en) | Vehicle detection method in foggy days based on deep learning | |
| CN107506703A (en) | A kind of pedestrian's recognition methods again for learning and reordering based on unsupervised Local Metric | |
| CN107273832B (en) | License plate recognition method and system based on integral channel characteristics and convolutional neural network | |
| KR101697161B1 (en) | Device and method for tracking pedestrian in thermal image using an online random fern learning | |
| CN108491498B (en) | Bayonet image target searching method based on multi-feature detection | |
| CN103530638B (en) | Method for pedestrian matching under multi-cam | |
| CN109635634B (en) | Pedestrian re-identification data enhancement method based on random linear interpolation | |
| CN106951899A (en) | Method for detecting abnormality based on image recognition | |
| CN111898627B (en) | SVM cloud microparticle optimization classification recognition method based on PCA | |
| CN111274926B (en) | Image data screening method, device, computer equipment and storage medium | |
| CN106919895A (en) | For the tracking and system of moving target | |
| CN114140665A (en) | Dense small target detection method based on improved YOLOv5 | |
| CN115841633A (en) | Power tower and power line associated correction power tower and power line detection method | |
| CN109615610B (en) | Medical band-aid flaw detection method based on YOLO v2-tiny | |
| CN111178405A (en) | Similar object identification method fusing multiple neural networks | |
| TWI696958B (en) | Image adaptive feature extraction method and its application | |
| CN113312999B (en) | High-precision detection method and device for diaphorina citri in natural orchard scene | |
| CN108985216B (en) | Pedestrian head detection method based on multivariate logistic regression feature fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder |
Address after: Room 1035, Back Block, Yuetian Building, No. 960, Jiefang North Road, Yuexiu District, Guangzhou City, Guangdong Province, 510400 Patentee after: Zhigang data service (Guangzhou) Co.,Ltd. Address before: Room 1035, Back Block, Yuetian Building, No. 960, Jiefang North Road, Yuexiu District, Guangzhou City, Guangdong Province, 510400 Patentee before: Guangwu Zhigang data service (Guangzhou) Co.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder |