CN112348894B - Method, system, equipment and medium for identifying position and state of scrap steel truck - Google Patents
Method, system, equipment and medium for identifying position and state of scrap steel truck Download PDFInfo
- Publication number
- CN112348894B CN112348894B CN202011210829.6A CN202011210829A CN112348894B CN 112348894 B CN112348894 B CN 112348894B CN 202011210829 A CN202011210829 A CN 202011210829A CN 112348894 B CN112348894 B CN 112348894B
- Authority
- CN
- China
- Prior art keywords
- scrap
- truck
- wagon
- trucks
- position information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The application provides a method, a system, equipment and a medium for identifying the position and the state of a scrap steel truck, which comprise the following steps: acquiring an image of a scrap truck under a current stacking and unloading scrap material area in a scrap plant; preprocessing the image to generate a data set, and training the data set by using a convolutional neural network to obtain a detection model for detecting the position of the scrap truck; calling the detection model to identify the position information of each scrap truck in the image to be detected; and comparing the identified position information of the scrap wagon with the position information of the region of interest, determining the type of the scrap wagon according to the appearance characteristics of the scrap wagon in the region of interest, and judging the current state of the scrap wagon. Compared with the prior art, the intelligent recognition device has the advantages of being high in efficiency and accuracy and providing powerful basis for follow-up automatic processing, and the position and the state of the scrap steel truck are intelligently recognized.
Description
Technical Field
The application belongs to the field of image processing, particularly relates to the field of metallurgy, and particularly relates to a method, a system, equipment and a medium for identifying the position and the state of a scrap steel truck.
Background
Steel mills have to store a large amount of scrap every year, and the scrap is transported mainly by trucks, for example, scrap dumps trucks and scrap dumps trucks are used to dump and unload in a designated area. The scrap steel stockpiling truck is a truck which is provided by a scrap steel plant in a unified way and has unified shape and appearance; the scrap steel unloading truck enters from an external channel, is different from a scrap steel stacking truck in shape and appearance and is a truck with various types.
In order to monitor the stacking and unloading conditions of the scrap trucks in real time, the positions of the scrap trucks are counted in a manual mode, and the states of the scrap trucks in a specified area are judged; however, the manual method has the disadvantages of large workload, complex operation and easy omission of statistics due to human factors.
Disclosure of Invention
In view of the above-mentioned shortcomings of the prior art, it is an object of the present application to provide a method, system, device and medium for identifying the position and status of a scrap truck, so as to ensure real-time monitoring of the condition of a scrap plant.
In order to achieve the above and other related objects, the present application provides a scrap steel wagon position and state identification method, including:
Acquiring an image of a scrap truck under a current stacking and unloading scrap material area in a scrap plant;
preprocessing the image to generate a data set, and training the data set by using a convolutional neural network to obtain a detection model for detecting the position of the scrap truck;
calling the detection model to identify the position information of each scrap truck in the image to be detected;
and comparing the identified position information of the scrap wagon with the position information of the region of interest, determining the type of the scrap wagon according to the appearance characteristics of the scrap wagon in the region of interest, and judging the current state of the scrap wagon.
An object of the application is to provide a steel scrap freight train position and state identification system, include:
the image acquisition module is used for acquiring an image of a scrap truck under a current stacking and unloading scrap material area in a scrap steel plant;
the detection model generation module is used for preprocessing the image to generate a data set and training the data set by utilizing a convolutional neural network to obtain a detection model for detecting the position of the scrap truck;
the position information identification module is used for calling the detection model to identify the position information of each scrap truck in the image to be detected;
and the state identification module is used for comparing the identified position information of the scrap wagon with the position information of the region of interest, determining the type of the scrap wagon according to the appearance characteristics of the scrap wagon in the region of interest, and judging the current state of the scrap wagon.
Another object of the present application is to provide an electronic device, comprising:
one or more processing devices;
a memory for storing one or more programs; when the one or more programs are executed by the one or more processing devices, the one or more processing devices are caused to execute the scrap wagon position and state identification method.
It is still another object of the present application to provide a computer-readable storage medium having stored thereon a computer program for causing the computer to execute the scrap wagon position and state identification method.
As mentioned above, the method, the system, the equipment and the medium for identifying the position and the state of the scrap steel truck have the following beneficial effects:
the method comprises the steps of acquiring an image containing a scrap truck under a current stacking and unloading scrap material area in a scrap steel plant based on machine vision, constructing a detection model for detecting position information of the scrap truck by utilizing a convolutional neural network, identifying the position information of the scrap truck and the position information of an interested area by comparison, and determining the type of the scrap truck by utilizing the appearance characteristics of the scrap truck, so as to determine the type, the number and the state of the scrap truck.
Drawings
FIG. 1 shows a flow chart of a scrap wagon position and condition identification method provided herein;
FIG. 2 is a view showing a setting scene of a camera device in a scrap steel plant according to the present invention;
FIG. 3 is a flowchart illustrating an embodiment of a scrap truck position and status identification method according to the present disclosure;
FIG. 4 is a block diagram illustrating a scrap truck position and status identification system according to the present disclosure;
fig. 5 is a schematic structural diagram of an electronic device provided in the present application.
Detailed Description
The following description of the embodiments of the present application is provided by way of specific examples, and other advantages and effects of the present application will be readily apparent to those skilled in the art from the disclosure herein. The present application is capable of other and different embodiments and its several details are capable of modifications and/or changes in various respects, all without departing from the spirit of the present application. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present application, and the drawings only show the components related to the present application and are not drawn according to the number, shape and size of the components in actual implementation, and the type, number and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.
Referring to fig. 1, a flowchart of a method for identifying a position and a state of a scrap wagon provided by the present application includes:
step S1, acquiring images of a scrap truck under a current stacking and unloading scrap material area in a scrap plant;
specifically, the scrap trucks comprise a scrap stacking truck and a scrap discharging truck, stacking and discharging are carried out in a specified site area (scrap stacking and discharging area), and images of the scrap trucks under the current scrap stacking and discharging area in a scrap plant are acquired through a camera device.
Step S2, preprocessing the image to generate a data set, and training the data set by using a convolutional neural network to obtain a detection model for detecting the position of the scrap truck;
step S3, calling the detection model to identify the position information of each scrap truck in the image to be detected;
and step S4, comparing the identified position information of the scrap trucks with the position information of the interested areas, determining the types of the scrap trucks according to the appearance characteristics of the scrap trucks in the interested areas, and judging the current states of the scrap trucks.
In the embodiment, the detection model is established through the convolutional neural network, the position information of each scrap truck in the image to be detected is identified through the detection model, the type of the scrap truck is determined according to the appearance characteristics of the scrap truck by comparing the identified position information of the scrap truck with the position information of the region of interest, and therefore the type, the number and the state of the scrap truck are determined.
Referring to fig. 3, a flowchart of an embodiment of a method for identifying a position and a state of a scrap truck according to the present application is detailed as follows:
on the basis of the above embodiment, before step S1, the method further includes:
and acquiring an image by utilizing a camera device positioned on the beam of the scrap steel plant, and setting a scrap steel material stacking and unloading area in the image as an interested area.
Specifically, a camera device, namely a camera or a camera, is arranged on a beam of a steel scrap factory, and a steel scrap stacking and unloading area in a camera picture is set as an identification interesting area according to a camera position and an actual scene; the cameras comprise cameras such as industrial cameras and IP (ball machine and gun) cameras, a plurality of cameras are arranged on the cross beam, a scene schematic diagram is shown in detail in figure 2, and a scene schematic diagram is set for the camera shooting device of the scrap steel material factory, so that the pictures shot by each camera cover the region of interest, namely, the region for stacking and unloading scrap steel materials, and the cameras can acquire images of scrap steel trucks under the region for unloading the scrap steel materials.
Wherein, the step S2 further includes:
collecting an image containing a scrap truck in a scrap material stacking and unloading area, and marking position information of the scrap truck in the image to form a data set;
inputting the images in the data set into a convolutional neural network based on deep learning for training, and establishing a detection model for detecting the position of the scrap truck; wherein the convolutional neural network comprises any one target detection model of SSD (Single Shot MultiBox Detector), yolo (you only look once) or fast-rcnn (RPN + CNN + ROI); the network comprises an SSD, yolo series, fast-rcnn and other object detection networks, and can be replaced by other object detection networks.
Specifically, an SSD target detection model is combined with a regression idea in the YOLO and an anchor mechanism in the Faster R-CNN, and multi-scale regional features of all positions of a full graph are used for regression, so that the characteristic of high YOLO speed is maintained, and the window prediction is more accurate as that of the Faster R-CNN. The method has the advantages that the YOLO target detection model is adopted to convert the target detection task into a regression problem, so that the detection speed is greatly increased, and the YOLO can process 45 images per second; meanwhile, the full graph information is used when each network predicts the target window, so that the false alarm rate is greatly reduced. The process of adopting the fast-rcnn target detection model based on deep learning target detection is more and more simplified, the identification precision is higher and higher, and the identification speed is higher and higher.
For example, the SSD destination detection model may be MobileNetV2-SSD deep learning neural network, wherein the MobileNetV2 network includes an inversion residual module (inversedresidual) for improving accuracy of image features and a Linear Bottleneck module (Linear bottleeck) for preventing information loss of a nonlinear function Relu, and the SSD network includes a base network and a pyramid network, wherein the base network may be transformed.
Specifically, in the MobileNetV2 network, the reverse residual module is mainly used to increase the extraction of image features to improve the accuracy, and the linear bottleneck module is mainly used to avoid the information loss of the nonlinear activation function ReLU. The network structure of MobileNetV2 includes: the inverse residual module Linear bottleeck includes a dimension layer for increasing dimension, a sampling layer for sampling, and an output layer for decreasing dimension. Firstly, in the dimension layer, the dimension layer increases the dimension of the image information from the k dimension to the tk dimension through the first convolution kernel and the activation function, for example, the dimension is increased from the k dimension to the tk dimension through the first convolution kernel conv of 1 × 1 and the activation function ReLU; then, in the sampling layer, the sampling layer samples the image information through a second convolution kernel and an activation function, for example, down-samples the image through a second convolution kernel conv of 3 × 3 and an activation function ReLU separable convolution (step size/stride > 1), when the characteristic dimension is already the tk dimension; finally, the output layer performs dimensionality reduction on the image information from the tk dimension to the k 'dimension by a third convolution kernel, e.g., from the tk to the k' dimension by a 1 × 1 third convolution kernel conv (no ReLU).
In addition, for the Linear Bottleneck module Linear bottleeck, in the neural network layer of the Linear Bottleneck module, when the step length of the convolution kernel is 1, connecting the input of the neural network layer to the output uses sum of elementwise to connect the input and output characteristics; when the step size is 2, then there is no shortcut connection input and output features.
In some implementations, a target detection algorithm of SSD (single-stage) can be used to predict targets with different frame sizes using feature maps of different scales. The SSD network comprises a basic network and a pyramid network, wherein the basic network can be changed, for example, the basic network of the SSD is a top 4 network of VGG-16, and the pyramid network is a simple convolution network with gradually-reduced feature maps and consists of 5 parts.
For another example, a MobileNetV2 network may replace VGG-16 in the original SSD network architecture, the configuration from Conv0 to Conv13 is adapted to MobileNetV2 model, and the last global average pooling, full connectivity layer and Softmax layer of MobileNetV2 are removed, and Conv6 and Conv7 may replace FC6 and FC7 of the original VGG-16, respectively. In some implementations, the MobileNetV2-SSD deep learning neural network is used to extract image feature output feature maps using MobileNetV2 network and then SSD object detection algorithms are used to detect information on multiple feature maps output by MobileNetV2 network.
Specifically, the content and format of the scrap steel truck position information are as follows:
[Truck xmin ，Truck ymin ，Truck xmax ，Truck ymax ]
wherein Truck xmin 、Truck ymin Respectively are x, y coordinates of the upper left corner of the rectangular target frame of the scrap steel Truck in the image, Truck xmax 、Truck ymax And the x and y coordinates of the lower right corner of the rectangular target frame of the scrap steel truck in the image are respectively.
In step S3, the established scrap steel truck target detection model is called, and the position information of each scrap steel truck target in the video is identified and recorded; after the model is called, the position information of all scrap trucks in the video image can be obtained, and the content and the format of the position information are as follows:
each row corresponds to a rectangular target frame of the scrap steel truck; truck1 xmin 、Truck1 ymin Respectively identifying x and y coordinates of the upper left corner point of the frame of the first scrap truck; truck1 xmax 、Truck1 ymax Respectively identifying x and y coordinates of a right lower corner point of the frame of the first scrap truck; TruckN is the Nth scrap truck identification frame.
Wherein, step 4 further comprises:
comparing the identified position information of the scrap wagon with the position information of the interested area, and classifying the scrap wagon according to the appearance characteristics of the scrap wagon in the interested area; the scrap steel truck with uniform shape and appearance in the appearance characteristics is a scrap steel stockpiling truck; the scrap trucks which are different from the scrap stacking trucks in shape and appearance and are various in types are scrap discharging trucks;
And judging the current state of the scrap trucks according to the number and the types of the scrap trucks in the region of interest.
The method comprises the following steps of comparing the position information of the identified scrap truck with the position information of a scrap pile unloading interested area, wherein the content and the format of the position information of the pile unloading interested area in an image are as follows:
[ROI xmin ，ROI ymin ，ROI xmax ，ROI ymax ]
wherein, ROI xmin 、ROI ymin Respectively the upper left x, y coordinates of the region of interest in the image, ROI xmax 、ROI ymax The x, y coordinates of the lower right corner of the region of interest in the image, respectively.
For each detected scrap Truck target, if the position information of the scrap Truck target meets the Truck xmin ＞ROI xmin ，Truck ymin ＞ROI ymin ,Truck xmax ＜ROI xmax ,Truck ymax ＜ROI ymax If so, judging that the target scrap wagon is positioned in the identification region of interest, and performing next classification treatment on the target scrap wagon; if not, not processing;
classifying the scrap trucks in the region of interest according to appearance characteristics, and dividing the scrap trucks into scrap trucks with uniform shapes and appearances D And a variety of types of trucks for discharging scrap of different shapes and appearances from the trucks for piling the scrap x . According to different appearance characteristics of the two trucks, the trucks can be classified and labeled. The label format is [ Class ]]I.e. target category, content of scrap dump Truck D Or the waste steel unloading Truck Truck x 。
Specifically, the specific method for classifying and implementing the scrap trucks is that each [ Truck ] in the image is classified according to the acquired position information of the scrap trucks in the region of interest xmin ，Truck ymin ，Truck xmax ，Truck ymax ]Partially intercepting to obtain an initial image of the scrap truck, and obtaining the image according to the scrapAnd (4) marking the truck category, and performing training analysis on the marked image to obtain a target classification model of the scrap truck.
And judging the state of the scrap trucks on the current picture according to the quantity and the type of the scrap trucks in the scrap dump interested area. The states of the method are mainly divided into:
wherein Flag is the state of scrap Truck in current picture, sum (Truck) D ) Number of scrap-stockpiling wagons, sum (Truck) present in the picture x ) The number of the scrap steel discharge trucks existing in the picture. The Flag is divided into 0-6 states, which respectively represent that no scrap truck exists, one scrap stacker truck exists, one scrap discharge truck exists, one scrap stacker truck and one scrap discharge truck exist, a plurality of scrap stacker trucks exist, a plurality of scrap discharge trucks exist, and a plurality of scrap stacker trucks and scrap discharge trucks exist in the scrap stacking and discharging area of interest.
Specifically, the obtained position information and the picture state of the scrap steel stacking truck and the scrap steel discharging truck are returned to an external equipment system for worker detection and subsequent identification.
In the embodiment, because the number and the type of the scrap trucks in the image of the video monitoring are not fixed, the type and the number of the scrap trucks in the region of interest can be effectively displayed by adopting the state representation mode, automatic stacking and discharging of a scrap steel material factory are realized, workers can grasp the stacking and discharging conditions in the site, and support is provided for subsequent automatic treatment.
Referring to fig. 4, a flowchart of a scrap steel wagon position and state identification system provided in the present application includes:
the image acquisition module 1 is used for acquiring an image of a scrap truck under a current scrap stacking and discharging area in a scrap plant;
before the image acquisition module, the method further comprises the following steps: and acquiring an image by utilizing a camera device positioned on the beam of the scrap steel plant, and setting a scrap steel material stacking and unloading area in the image as an interested area.
The detection model generation module 2 is used for preprocessing the image to generate a data set, and training the data set by using a convolutional neural network to obtain a detection model for detecting the position of the scrap truck;
specifically, collecting an image containing a scrap truck in a stacking and unloading scrap material area, and marking position information of the scrap truck in the image to form a data set;
Inputting the images in the data set into a convolutional neural network based on deep learning for training, and establishing a detection model for detecting the position of the scrap truck; wherein the convolutional neural network comprises any one of SSD, yolo, or fast-rcnn.
The position information identification module 3 is used for calling the detection model to identify the position information of each scrap truck in the image to be detected;
specifically, the detection model is called to identify the position information of each scrap truck in the image to be detected, and the content and format of the position information are as follows:
in the formula, each row corresponds to a rectangular target frame of the scrap steel truck; truck1 xmin 、Truck1 ymin Respectively identifying x and y coordinates of the upper left corner point of the frame of the first scrap truck; truck1 xmax 、Truck1 ymax Respectively identifying x and y coordinates of a right lower corner point of the frame of the first scrap truck; TruckN is the Nth scrap truck identification frame.
The state identification module 4 is used for comparing the identified position information of the scrap wagon with the position information of the region of interest, determining the type of the scrap wagon according to the appearance characteristics of the scrap wagon in the region of interest, and judging the current state of the scrap wagon;
comparing the identified position information of the scrap wagon with the position information of the interested area, and classifying the scrap wagon according to the appearance characteristics of the scrap wagon in the interested area; the scrap steel truck with uniform shape and appearance in the appearance characteristics is a scrap steel stockpiling truck; the scrap trucks which are different from the scrap stacking trucks in shape and appearance and are various in types are scrap discharging trucks;
The format and the content of the position information of the region of interest are as follows: [ ROI xmin ，ROI ymin ，ROI xmax ，ROI ymax ](ii) a In the formula, ROI xmin 、ROI ymin Respectively the upper left x, y coordinates of the region of interest in the image, ROI xmax 、ROI ymax The x and y coordinates of the lower right corner of the interested area in the image are respectively;
comparing the identified position information of the scrap steel Truck with the position information of the region of interest, and if the position information meets the Truck xmin ＞ROI xmin ,Truck ymin ＞ROI ymin ,Truck xmax ＜ROI xmax ,Truck ymax ＜ROI ymax Judging that the scrap wagon is in the region of interest, classifying and determining the type of the scrap wagon according to the appearance characteristics of the scrap wagon, and when the shape and the appearance in the appearance characteristics are uniform, determining that the scrap wagon is a scrap stacking wagon Truck D (ii) a And when the appearance characteristics are different from the shapes and appearances of the scrap steel stacking trucks and the types of the scrap steel stacking trucks are various, the scrap steel trucks are scrap steel unloading trucks.
And judging the current state of the scrap trucks according to the number and the types of the scrap trucks in the region of interest.
Wherein Flag is the current scrap Truck status, sum (Truck) D ) Number of trucks for scrap stockpiling, sum (Truck) x ) The number of trucks for discharging the scrap steel. The Flag is divided into 0-6 states, which respectively represent that no scrap truck exists, one scrap stacking truck exists, one scrap discharging truck exists, one scrap stacking truck and one scrap discharging truck exist, a plurality of scrap stacking trucks exist, a plurality of scrap discharging trucks exist, and a plurality of scrap discharging trucks exist simultaneously in the scrap stacking and discharging area of interest A scrap dump truck and a scrap dump truck.
It should be noted that the identification method of the position and the state of the scrap wagon and the identification system of the position and the state of the scrap wagon are in a one-to-one correspondence relationship, and here, the technical details and the technical effects related to the identification system of the position and the state of the scrap wagon are the same as those of the identification method, and are not repeated herein, please refer to the identification method of the position and the state of the scrap wagon.
Referring now to fig. 5, a schematic diagram of an electronic device (e.g., a terminal device or server 500) suitable for implementing embodiments of the present disclosure is shown, where the terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet), a PMP (portable multimedia player), a vehicle terminal (e.g., a car navigation terminal), etc., and a fixed terminal such as a digital TV, a desktop computer, etc. the electronic device shown in fig. 5 is only one example and should not bring any limitations to the functions and scope of use of the embodiments of the present disclosure.
As shown in fig. 5, electronic device 500 may include a processing means (e.g., central processing unit, graphics processor, etc.) 501 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage means 508 into a Random Access Memory (RAM) 503. In the RAM503, various programs and data necessary for the operation of the electronic apparatus 500 are also stored. The processing device 501, the ROM502, and the RAM503 are connected to each other through a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
Generally, the following devices may be connected to the I/O interface 505: input devices 506 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 507 including, for example, a Liquid Crystal Display (LCD), speakers, vibrators, and the like; storage devices 508 including, for example, magnetic tape, hard disk, etc.; and a communication device 509. The communication means 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. While fig. 5 illustrates an electronic device 500 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.
In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 509, or installed from the storage means 508, or installed from the ROM 502. The computer program performs the above-described functions defined in the methods of the embodiments of the present disclosure when executed by the processing device 501.
The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: the method of the above-described steps S1 to S4 is performed.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
To sum up, this application contains the image of steel scrap freight train under the current pile-up and unload steel scrap material region in based on machine vision gathers the steel scrap material factory, utilizes convolution neural network to establish the detection model who detects steel scrap freight train positional information, through the positional information contrast of contrast discernment steel scrap freight train and the positional information contrast of the region of interest, utilizes the appearance characteristic of steel scrap freight train to confirm the type of steel scrap freight train to confirm steel scrap freight train type, quantity and state, compare prior art, realized the position and the state of intelligent discernment steel scrap freight train, have efficient, the advantage of high rate of accuracy, provide powerful foundation for follow-up automated processing.
The above embodiments are merely illustrative of the principles and utilities of the present application and are not intended to limit the application. Any person skilled in the art can modify or change the above-described embodiments without departing from the spirit and scope of the present application. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical concepts disclosed in the present application shall be covered by the claims of the present application.
Claims (9)
1. A method for identifying the position and the state of a scrap steel truck is characterized by comprising the following steps:
acquiring an image of a scrap truck under a current stacking and unloading scrap material area in a scrap plant;
preprocessing the image to generate a data set, and training the data set by using a convolutional neural network to obtain a detection model for detecting the position of the scrap truck;
calling the detection model to identify the position information of each scrap truck in the image to be detected;
comparing the identified position information of the scrap wagon with the position information of the region of interest, determining the type of the scrap wagon according to the appearance characteristics of the scrap wagon in the region of interest, and judging the current state of the scrap wagon; the format and the content of the position information of the region of interest are as follows: [ RoI xmin ，ROI ymin ，ROI xmax ，ROI ymax ]
Wherein, ROI xmin 、ROI ymin Respectively the upper left corner x, y coordinates, ROI of the region of interest in the image xmax 、ROI ymax The x and y coordinates of the lower right corner of the region of interest in the image are respectively;
comparing the identified position information of the scrap steel Truck with the position information of the region of interest, and if the position information meets the Truck xmin ＞ROI xmin ，Truck ymin ＞ROI ymin ，Truck xma x＜ROI xmax ，Truck ymax ＜ROI ymax Judging that the scrap wagon is in the region of interest, classifying and determining the type of the scrap wagon according to the appearance characteristics of the scrap wagon, and when the shape and the appearance in the appearance characteristics are uniform, determining that the scrap wagon is a scrap stacking wagon Truck D (ii) a And when the appearance characteristics are different from the shapes and appearances of the scrap steel stacking trucks and the types of the scrap steel stacking trucks are various, the scrap steel trucks are scrap steel unloading trucks.
2. The method of claim 1, wherein the step of collecting the image of the scrap wagon contained in the area of the scrap wagon currently loaded and unloaded from the scrap plant further comprises: and acquiring an image by utilizing a camera device positioned on the beam of the scrap steel plant, and setting a scrap steel material stacking and unloading area in the image as an interested area.
3. The scrap truck position and condition identification method according to claim 1, wherein the preprocessing the image to generate a data set, and the step of training the data set using a convolutional neural network to obtain a detection model for detecting scrap truck position comprises:
Collecting an image containing a scrap truck in a scrap material stacking and unloading area, and marking position information of the scrap truck in the image to form a data set;
inputting the images in the data set into a convolutional neural network based on deep learning for training, and establishing a detection model for detecting the position of the scrap truck; wherein the convolutional neural network comprises any one of SSD, yolo or faster-rcnn.
4. The method for identifying the position and the state of the scrap trucks according to claim 1, 2 or 3, wherein the step of calling the detection model to identify the position information of each scrap truck in the image to be detected comprises the following steps: calling the detection model to identify the position information of each scrap truck in the image to be detected, wherein the content and the format of the position information are as follows:
in the formula, each row corresponds to a rectangular target frame of the scrap steel truck; truck1 xmin 、Truck1 ymin Respectively identifying x and y coordinates of the upper left corner point of the frame of the first scrap truck; truck1 xmax 、Truck1 ymax Respectively identifying x and y coordinates of a right lower corner point of the frame of the first scrap truck; TruckN is Nth scrap truck identificationAnd (5) framing.
5. The method according to claim 1, wherein the step of comparing the identified position information of the scrap trucks with the position information of the area of interest, determining the type of the scrap trucks according to the appearance characteristics of the scrap trucks in the area of interest, and determining the current state of the scrap trucks comprises:
Comparing the identified position information of the scrap wagon with the position information of the interested area, and classifying the scrap wagon according to the appearance characteristics of the scrap wagon in the interested area; the scrap steel truck with uniform shape and appearance in the appearance characteristics is a scrap steel stockpiling truck; the scrap trucks which are different from the scrap stacking trucks in shape and appearance and are various in types are scrap discharging trucks;
and judging the current state of the scrap trucks according to the number and the types of the scrap trucks in the region of interest.
6. The method of claim 5, wherein the step of determining the current status of the scrap trucks based on the number and type of scrap trucks within the area of interest comprises:
wherein Flag is the current scrap Truck status, Sum (Truck) D ) Number of trucks for scrap stockpiling, sum (Truck) x ) The number of trucks discharging the scrap steel; the Flag is divided into 0-6 states, which respectively represent that no scrap truck exists, one scrap stacker truck exists, one scrap discharge truck exists, one scrap stacker truck and one scrap discharge truck exist, a plurality of scrap stacker trucks exist, a plurality of scrap discharge trucks exist, and a plurality of scrap stacker trucks and scrap discharge trucks exist in the scrap stacking and discharging area of interest.
7. The utility model provides a scrap steel freight train position and state identification system which characterized in that includes:
the image acquisition module is used for acquiring an image of a scrap truck under a current stacking and unloading scrap material area in a scrap steel plant;
the detection model generation module is used for preprocessing the image to generate a data set and training the data set by utilizing a convolutional neural network to obtain a detection model for detecting the position of the scrap truck;
the position information identification module is used for calling the detection model to identify the position information of each scrap truck in the image to be detected;
the state identification module is used for comparing the identified position information of the scrap wagon with the position information of the region of interest, determining the type of the scrap wagon according to the appearance characteristics of the scrap wagon in the region of interest, and judging the current state of the scrap wagon; the format and the content of the position information of the region of interest are as follows:
[ROI xmin ，ROI ymin ，ROI xmax ，ROI ymax ]wherein, ROI xmin 、ROI ymin Respectively the upper left x, y coordinates of the region of interest in the image, ROI xmax 、ROI ymax The x and y coordinates of the lower right corner of the interested area in the image are respectively;
comparing the identified position information of the scrap steel Truck with the position information of the region of interest, and if the position information meets the Truck xmin ＞ROI xmin ，Truck ymin ＞ROI ymin ，Truck xmax ＜ROI xmax ，Truck ymax ＜ROI ymax Judging that the scrap wagon is in the region of interest, classifying and determining the type of the scrap wagon according to the appearance characteristics of the scrap wagon, and when the shape and the appearance in the appearance characteristics are uniform, determining that the scrap wagon is a scrap stacking wagon Truck D (ii) a And when the appearance characteristics are different from the shapes and appearances of the scrap steel stacking trucks and the types of the scrap steel stacking trucks are various, the scrap steel trucks are scrap steel unloading trucks.
8. An electronic device, characterized in that: the method comprises the following steps:
one or more processing devices;
a memory for storing one or more programs; when the one or more programs are executed by the one or more processing devices, the one or more processing devices are caused to implement the scrap wagon position and status identification method of any one of claims 1 to 6.
9. A computer-readable storage medium having stored thereon a computer program for causing a computer to execute the scrap wagon position and state recognition method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011210829.6A CN112348894B (en) | 2020-11-03 | 2020-11-03 | Method, system, equipment and medium for identifying position and state of scrap steel truck |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011210829.6A CN112348894B (en) | 2020-11-03 | 2020-11-03 | Method, system, equipment and medium for identifying position and state of scrap steel truck |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112348894A CN112348894A (en) | 2021-02-09 |
| CN112348894B true CN112348894B (en) | 2022-07-29 |
Family
ID=74356868
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011210829.6A Active CN112348894B (en) | 2020-11-03 | 2020-11-03 | Method, system, equipment and medium for identifying position and state of scrap steel truck |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112348894B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113743210A (en) * | 2021-07-30 | 2021-12-03 | 阿里巴巴达摩院(杭州)科技有限公司 | Image recognition method and scrap grade recognition method |
| CN114860359B (en) * | 2022-04-06 | 2024-05-14 | 阿里巴巴达摩院(杭州)科技有限公司 | Image processing method, system and storage medium |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101692628B1 (en) * | 2014-12-24 | 2017-01-04 | 한동대학교 산학협력단 | Method for detecting right lane area and left lane area of rear of vehicle using region of interest and image monitoring system for vehicle using the same |
| CN108073864B (en) * | 2016-11-15 | 2021-03-09 | 北京市商汤科技开发有限公司 | Target object detection method, device and system and neural network structure |
| CN107766789B (en) * | 2017-08-21 | 2020-05-29 | 浙江零跑科技有限公司 | Vehicle detection positioning method based on vehicle-mounted monocular camera |
| CN107633505A (en) * | 2017-08-24 | 2018-01-26 | 南京理工大学 | A kind of undercarriage detection method based on target gray distribution character |
| US10614318B1 (en) * | 2019-10-25 | 2020-04-07 | 7-Eleven, Inc. | Sensor mapping to a global coordinate system using a marker grid |
| CN109829386B (en) * | 2019-01-04 | 2020-12-11 | 清华大学 | Intelligent vehicle passable area detection method based on multi-source information fusion |
| CN110070083A (en) * | 2019-04-24 | 2019-07-30 | 深圳市微埃智能科技有限公司 | Image processing method, device, electronic equipment and computer readable storage medium |
| CN111079747B (en) * | 2019-12-12 | 2020-10-09 | 哈尔滨市科佳通用机电股份有限公司 | Railway wagon bogie side frame fracture fault image identification method |
| CN111242088B (en) * | 2020-01-22 | 2023-11-28 | 上海商汤临港智能科技有限公司 | Target detection method and device, electronic equipment and storage medium |
| CN111325146B (en) * | 2020-02-20 | 2021-06-04 | 吉林省吉通信息技术有限公司 | Truck type and axle type identification method and system |
| CN111340027A (en) * | 2020-03-05 | 2020-06-26 | 中冶赛迪重庆信息技术有限公司 | Steel pile identification method and system, electronic equipment and medium |
-
2020
- 2020-11-03 CN CN202011210829.6A patent/CN112348894B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN112348894A (en) | 2021-02-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108230357B (en) | Key point detection method and device, storage medium and electronic equipment | |
| CN112348894B (en) | Method, system, equipment and medium for identifying position and state of scrap steel truck | |
| CN111582263A (en) | License plate recognition method and device, electronic equipment and storage medium | |
| CN112669344A (en) | Method and device for positioning moving object, electronic equipment and storage medium | |
| CN108647587B (en) | People counting method, device, terminal and storage medium | |
| CN111652869B (en) | Slab void identification method, system, medium and terminal based on deep learning | |
| CN110659391A (en) | Video detection method and device | |
| CN109711427A (en) | Object detection method and Related product | |
| CN111274934A (en) | Implementation method and system for intelligently monitoring forklift operation track in warehousing management | |
| CN111291587A (en) | Pedestrian detection method based on dense crowd, storage medium and processor | |
| CN115311632A (en) | Vehicle weight recognition method and device based on multiple cameras | |
| CN115600953A (en) | Monitoring method and device for warehouse positions, computer equipment and storage medium | |
| CN113065379B (en) | Image detection method and device integrating image quality and electronic equipment | |
| CN111191556A (en) | Face recognition method and device and electronic equipment | |
| CN113470065B (en) | Moving object detection and tracking method and apparatus | |
| CN117422905A (en) | Target identification method, device, equipment and storage medium | |
| CN111310595B (en) | Method and device for generating information | |
| CN110222576B (en) | Boxing action recognition method and device and electronic equipment | |
| CN114022658A (en) | Target detection method, device, storage medium and terminal | |
| CN114550102A (en) | Cargo accumulation detection method, device, equipment and system | |
| CN113869123A (en) | Crowd-based event detection method and related device | |
| CN112560700A (en) | Information association method and device based on motion analysis and electronic equipment | |
| CN111428632A (en) | Winch state detection method and miner behavior detection method | |
| CN111724337B (en) | Cold bed top punching identification method and system, electronic equipment and medium | |
| CN113435352B (en) | Civilized city scoring method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: 401329 No. 5-6, building 2, No. 66, Nongke Avenue, Baishiyi Town, Jiulongpo District, Chongqing Patentee after: MCC CCID information technology (Chongqing) Co.,Ltd. Address before: 20-24 / F, No.7 Longjing Road, North New District, Yubei District, Chongqing Patentee before: CISDI CHONGQING INFORMATION TECHNOLOGY Co.,Ltd. |
|
| CP03 | Change of name, title or address |