CN117764468A - Intelligent loading and plugging line control method and system based on Internet of things and machine vision - Google Patents
Intelligent loading and plugging line control method and system based on Internet of things and machine vision Download PDFInfo
- Publication number
- CN117764468A CN117764468A CN202311662983.0A CN202311662983A CN117764468A CN 117764468 A CN117764468 A CN 117764468A CN 202311662983 A CN202311662983 A CN 202311662983A CN 117764468 A CN117764468 A CN 117764468A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- transport vehicle
- port
- loading
- line
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000009471 action Effects 0.000 claims abstract description 43
- 239000000463 material Substances 0.000 claims abstract description 41
- 238000001514 detection method Methods 0.000 claims abstract description 23
- 230000000007 visual effect Effects 0.000 claims abstract description 18
- 238000013136 deep learning model Methods 0.000 claims abstract description 16
- 230000004888 barrier function Effects 0.000 claims description 22
- 238000004891 communication Methods 0.000 claims description 14
- 238000004458 analytical method Methods 0.000 claims description 11
- 238000013527 convolutional neural network Methods 0.000 claims description 9
- 238000013135 deep learning Methods 0.000 claims description 7
- 238000012544 monitoring process Methods 0.000 claims description 7
- 230000003993 interaction Effects 0.000 claims description 6
- 238000003032 molecular docking Methods 0.000 claims description 4
- 238000007726 management method Methods 0.000 description 27
- 238000012545 processing Methods 0.000 description 12
- 238000005516 engineering process Methods 0.000 description 8
- 238000012795 verification Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 238000004590 computer program Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000002159 abnormal effect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000015654 memory Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 239000000428 dust Substances 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000012015 optical character recognition Methods 0.000 description 3
- 238000012384 transportation and delivery Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000007599 discharging Methods 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000007619 statistical method Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000005303 weighing Methods 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000003466 anti-cipated effect Effects 0.000 description 1
- 230000003416 augmentation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 239000004557 technical material Substances 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Landscapes
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The invention provides a control method of an intelligent loading line based on the Internet of things and machine vision, which comprises the following steps: s1, detecting vehicle position information and vehicle identity information of a transport vehicle entering a designated area of a port loading line, and executing S2 when the vehicle position information and the vehicle identity information meet preset conditions; s2, identifying the action state of a hopper and the residual state of hopper materials of the transport vehicle in a visual detection mode, and then executing S3; and S3, judging whether the transport vehicle is allowed to leave the designated area of the port loading refute line or not according to the hopper action state and the hopper material residual state of the transport vehicle. The invention further provides a control system of the intelligent loading line based on the Internet of things and the machine vision, which comprises a port operation management platform, a deep learning model, visual equipment and a vehicle information identification subsystem. According to the invention, the efficient intelligent detection and control of port loading and unloading lines are realized according to the actions of materials and hoppers in the port transport vehicle.
Description
Technical Field
The invention relates to the technical field of intelligent line installation equipment and port software, in particular to a control method and a system for intelligent line installation based on the Internet of things and machine vision.
Background
With the improvement of computer performance, the progress of sensor technology and the innovation of algorithms, the machine vision technology is rapidly developed, and the method has wide application prospect in various fields. At present, the operation of the loading and unloading line platform of most ports is that site operation personnel such as dumper drivers, warehouse management staff and the like observe, judge, command, count and record in a manual mode, data acquisition, information verification, safety detection and related prompts are not carried out on the operation, the conditions that the loading and unloading platform is not unloaded, is not unloaded completely, is not put down a hopper after unloading, is unloaded and is loaded in error, is not loaded with weight and is directly unloaded, is loaded with weight and is unloaded repeatedly are easily caused, potential safety hazards exist in the wrong operation, the operation cargo amount is inconsistent, cargo delivery fails, and loss is brought to a cargo owner and ports. The prior art mainly relies on drivers and tally operators to observe, judge and command in a manual mode, whether weighing is completed, whether a specified loading and unloading platform is entered, whether unloading can be carried out or whether unloading is completed or not is completely unloaded, links such as unloading is completed or unloading is not completed, links such as leaving the loading and unloading platform are separated, and operation information and a handover list are manually counted and recorded. The prior art also has the following drawbacks and problems:
1. the port operation is busy, and the vehicles have errors caused by driver errors and the loading errors due to the fact that the vehicles are loaded on the loading platforms by mistake, so that great losses are caused to port parties, cargo parties and ship parties;
2. the vehicle does not operate according to a standard flow, is not weighted, or is loaded on a loading and unloading platform for unloading after being weighted repeatedly, so that the delivery cargo quantity of a harbor party and a shipside is inconsistent, and the condition of overload of a barge can occur;
3. the vehicles are not unloaded after being loaded on the docking platform, or leave the platform without being unloaded cleanly, and the delivery cargo quantity of the port party and the ship party is inconsistent;
4. when the vehicle finishes unloading and leaves the platform, the situation that the vehicle hopper is not dropped exists, and great potential safety hazards exist, so that the equipment in the harbor is damaged and the running safety of a driver of the vehicle is caused;
5. the field operation environment is not friendly to workers, including noise, dust, shoreside, night shifts and the like, and the technology is less affected by the environment.
Although the Chinese patent application No. 201810060393.3 discloses a dynamic intelligent management and control system for unloading and entering and exiting operations of bulk grain automobiles, the invention is not applicable to port loading and unloading line platforms, and is particularly not applicable to loading and unloading lines provided with port operation management platforms.
Disclosure of Invention
In view of the above, it is necessary to provide a control method and system for intelligent port loading and unloading line based on internet of things and machine vision, so as to overcome the drawbacks in the background art, and solve the technical problem of how to realize efficient intelligent detection and control of port loading and unloading line according to the material and hopper actions in the port transport vehicle.
Abbreviations:
edge calculation: (Edge Computing) is a distributed Computing architecture, where the Computing of applications, data and services is handled by hub nodes, moving to Edge nodes on the network logic, splitting the large services handled by the hub nodes entirely, cutting into smaller and more manageable parts, and distributing to the Edge nodes for processing;
machine vision: (Machine Vision) refers to a technique of acquiring, processing, analyzing, and understanding an image or video with a camera or other sensor; the method combines knowledge and technology in multiple fields such as computer vision, image processing, pattern recognition, machine learning and the like;
cloud service: (Cloud Service) is a network-related Service-based model of augmentation, use, and interaction that can provide dynamic, easily-scalable, virtualized resources over the internet;
laser radar: (LiDAR) is a technique that uses a laser beam to measure distance and obtain environmental geometry, structure, and motion information; the method comprises the steps of emitting a high-power short-pulse laser beam, and measuring the round trip time of the laser beam from an emitting point to the surface of a target object, so as to calculate the distance and the position of the target object;
the Internet of things: the internet of things (IoT), which is an extended and expanded network based on the internet, combines various information sensing devices with the network to form a huge network, and realizes the interconnection and intercommunication of people, machines and objects at any time and any place;
optical character recognition: (OCR) refers to a technology for analyzing, identifying and processing an image file containing text data to obtain text and layout information;
installing a connection line: one of the port loading barges is published in electric power equipment 2018, 32 nd stage, "optimizing and modifying a loading line loading capacity improvement", and other port loading barges for directly or indirectly loading cargoes to the barge by using mechanical equipment in the prior art.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
the invention provides a control method of an intelligent loading line based on the Internet of things and machine vision, which comprises the following steps:
s1, detecting vehicle position information and vehicle identity information of a transport vehicle entering a designated area of a port loading line, and executing S2 when the vehicle position information and the vehicle identity information meet preset conditions;
s2, identifying the action state of a hopper and the residual state of hopper materials of the transport vehicle in a visual detection mode, and then executing S3;
and S3, judging whether the transport vehicle is allowed to leave the designated area of the port loading refute line or not according to the hopper action state and the hopper material residual state of the transport vehicle.
Further, between S1 and S2, the control method further includes the following steps:
s102, judging whether the transport vehicle is in a heavy weight state, if so, executing S2, and if not, sending out prompt information.
Further, in S1, vehicle position information is detected by means of laser radar detection for a transport vehicle entering a designated area of a port loading line.
Further, in S2, the hopper material residual state is detected by applying a deep convolutional neural network based on deep learning.
Further, in S3, when the transport vehicle completes the bucket lifting and dropping actions and the hopper has no material residue, the transport vehicle is allowed to travel away from the designated area of the port loading line; in S3, when the transport vehicle finishes the bucket lifting action but does not finish the bucket falling action, the transport vehicle is not allowed to leave the appointed area of the port loading refute line; in S3, when the transport vehicle does not complete the bucket lifting action or the bucket falling action, the transport vehicle is not allowed to leave the appointed area of the port loading refute line; in S3, when the transport vehicle completes the bucket lifting and dropping actions and the hopper has material residues, the transport vehicle is not allowed to leave the designated area of the port loading line.
Further, in S3, when the transport vehicle is not allowed to drive away from the designated area of the port loading line, an alarm prompt is sent;
in S3, when the transport vehicle is not allowed to drive away from the appointed area of the port loading refute line, controlling the barrier gate equipment in the port loading refute line not to open the gate;
in S1, when any one of the vehicle position information and the vehicle identity information does not satisfy the preset condition, an alarm prompt is sent.
Further, before S1, the method further includes the following steps:
s100, controlling the barrier equipment in the port loading refute line to open a barrier, and allowing the transport vehicle to drive to the appointed area of the port loading refute line through the barrier equipment.
The invention provides a control system of an intelligent loading line based on the Internet of things and machine vision, which comprises a port operation management platform, a deep learning model, visual equipment and a vehicle information identification subsystem;
if the deep learning model is embedded in the port operation management platform, the port operation management platform is respectively in communication connection with the vision equipment and the vehicle information identification subsystem;
if the deep learning model is arranged on an independent intelligent analysis platform, the independent intelligent analysis platform performs information interaction with a port operation management platform, the independent intelligent analysis platform is in communication connection with visual equipment, and the port operation management platform is in communication connection with a vehicle information identification subsystem;
the port operation management platform is used for managing, monitoring and analyzing port loading lines and transport vehicles; the port operation management platform judges whether the transport vehicle is allowed to leave a designated area of a port loading line or not according to the action state of a hopper of the transport vehicle and the residual state of hopper materials;
the deep learning model is used for analyzing the residual state of the hopper materials by using a deep convolutional neural network;
the visual equipment is used for visually detecting the action state of the hopper of the transport vehicle and the residual state of the hopper material;
the vehicle information identification subsystem is used for detecting vehicle position information and vehicle identity information of a transport vehicle entering a designated area of the port loading line.
Further, the vehicle information recognition subsystem comprises a vehicle position information recognition device and a vehicle identity information recognition device; the port operation management platform is respectively in communication connection with the vehicle position information identification equipment and the vehicle identity information identification equipment;
the vehicle position information identification device is used for detecting vehicle position information of a transport vehicle entering a designated area of the port loading line;
the vehicle identity information identification equipment is used for detecting vehicle identity information of a transport vehicle entering a designated area of the port loading line;
the port job management platform also performs the following control logic:
when the vehicle position information and the vehicle identity information meet preset conditions, the hopper action state and the hopper material residual state of the transport vehicle are identified in a visual detection mode.
The invention still further proposes a computer readable storage medium storing program code for execution, the program code comprising instructions for implementing the method as claimed in any one of the preceding claims.
The beneficial effects of the invention are as follows:
the invention relates to an intelligent reconstruction design of a line for assembling and connecting based on the combination of artificial intelligence, machine vision and the technology of the Internet of things, which relates to the artificial intelligence technology such as deep learning, computer vision, image recognition and the like, and the machine vision technology such as a camera, an image processing algorithm, a machine control system and the like, and realizes the automatic recognition, analysis and processing of objects, scenes and image data of the line for assembling and connecting. The invention can automatically realize operations such as vehicle license plate recognition, vehicle operation verification, vehicle discharging detection, discharging condition monitoring, vehicle hopper falling state analysis, vehicle departure prompting and the like based on machine vision and edge calculation, realizes intelligent loading and unloading line operation, and provides remote monitoring and remote emergency intervention to complete special condition treatment.
The invention also has the following advantages:
(1) The intelligent loading platform is established by combining the Internet of things with machine vision, so that the remote control supervision of the platform is realized;
(2) Various operation information is identified, captured, collected and processed through the intelligent line loading system, and a manual mode is replaced by a technical mode, so that irregular operation is reduced, the operation efficiency is improved, potential safety hazards are eliminated, the labor cost is reduced, and the freight quality is improved;
(3) Reducing the edge calculated amount under the condition of decreasing reasonable control precision through a quantized deep learning model;
(4) The identity and the state of each vehicle are accurately identified through intelligent license plate identification;
(5) The vehicle, the loading platform, the operation instruction and other related information management are carried out through cloud service, the loading system realizes the calibration of the vehicle identity and the operation instruction, the statistical analysis is carried out on the operation data, and the operation error is avoided;
(6) Through license plate recognition, state judgment, vehicle action recognition, edge calculation, internet of things and cloud service combination, full-flow supervision of assembly line operation is realized, and vehicles are ensured to operate according to a standard flow.
Drawings
The accompanying drawings are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification. The drawings illustrate exemplary embodiments of the invention and, together with the description, serve to explain the principles of the invention. These figures are for illustration only and thus are not limiting of the invention.
FIG. 1 is a workflow diagram of a control method of an intelligent loading line based on the Internet of things and machine vision of the present invention;
fig. 2 is a schematic structural diagram of a control system of an intelligent loading line based on internet of things and machine vision according to the embodiment 2 of the present invention;
fig. 3 is a schematic structural diagram of another control system of an intelligent refuting line based on internet of things and machine vision according to embodiment 3 of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be further clearly and completely described in the following in conjunction with the embodiments of the present invention. It should be noted that the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terms "first," "second," "third," "fourth," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a definition of "a first", "a second", "a third" or a fourth "feature may explicitly or implicitly include one or more of such features.
The following is a detailed description of embodiments of the invention depicted in the accompanying drawings. The embodiments are in detail in order to clearly communicate the invention. However, the amount of detail offered is not intended to limit the anticipated variations of embodiments; on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined by the appended claims.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the invention. It will be apparent to one skilled in the art that embodiments of the invention may be practiced without some of these specific details.
Embodiments of the present invention include various steps, which will be described below. These steps may be performed by hardware components or may be embodied in machine-executable instructions, which may be used in a general-purpose or special-purpose processor programmed with the instructions to perform the steps. Alternatively, the steps may be performed by a combination of hardware, software, and firmware, and/or a human operator.
The various methods described herein may be practiced by combining one or more machine-readable storage media containing code according to the present invention with appropriate standard computer hardware to execute the code contained therein. An apparatus for practicing various embodiments of the invention could include one or more computers (or one or more processors within a single computer) and a storage system containing or having network access to a computer program encoded in accordance with the various methods described herein, and method steps of the invention could be accomplished by modules, routines, subroutines, or sub-portions of a computer program product.
If the specification states a component or feature "may", "could" or "could" include or have a feature, that particular component or feature need not be included.
As used in the specification herein and in the claims that follow, the meaning of "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Furthermore, as used in the description herein, the meaning of "in" includes "in" and "on" unless the context clearly dictates otherwise.
Exemplary embodiments will now be described more fully hereinafter with reference to the accompanying drawings, in which exemplary embodiments are shown. These exemplary embodiments are provided for illustrative purposes only and to complete and convey the scope of the invention to those skilled in the art. The disclosed invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Various modifications will be apparent to those skilled in the art. The general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the invention. Furthermore, all statements herein reciting embodiments of the invention, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future (i.e., any elements developed that perform the same function, regardless of structure). Also, the terminology and phraseology used is for the purpose of describing the exemplary embodiments and should not be regarded as limiting. Thus, the invention is to be accorded the widest scope encompassing numerous alternatives, modifications and equivalents consistent with the principles and features disclosed. For the sake of clarity, details of technical material that is known in the technical fields related to the invention have not been described in detail so that the invention is not unnecessarily obscured.
Thus, for example, it will be appreciated by those skilled in the art that diagrams, schematics, illustrations, and the like represent conceptual views or processes embodying the system and method of the invention. The functions of the various elements shown in the figures may be provided through the use of dedicated hardware as well as hardware capable of executing associated software. Similarly, any switches shown in the figures are conceptual only. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the entity embodying the invention. Those of ordinary skill in the art will further appreciate that the exemplary hardware, software, processes, methods, and/or operating systems described herein are for illustrative purposes and are not intended to be limited to any particular named element.
Embodiments of the invention may be provided as a computer program product that may include a machine-readable storage medium tangibly embodying instructions thereon, which may be used to program a computer (or other electronic devices) to perform processing. The term "machine-readable storage medium" or "computer-readable storage medium" includes, but is not limited to, fixed (hardware) drives, magnetic tapes, floppy diskettes, optical disks, compact disk read-only memories (CD-ROMs)), and magneto-optical disks, semiconductor memories. A computer program product may include code and/or machine-executable instructions that may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.
Furthermore, embodiments may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program code or code segments (e.g., a computer program product) to perform the necessary tasks may be stored in a machine readable medium. The processor may perform the necessary tasks.
The systems depicted in some of the figures may be provided in various configurations. In some embodiments, the system may be configured as a distributed system, wherein one or more components of the system are distributed over one or more networks in the cloud computing system.
Each of the appended claims defines a separate invention which, for infringement purposes, is recognized as including equivalents to the various elements or limitations specified in the claims. Depending on the context, all references below to "invention" may in some cases refer to only certain specific embodiments. In other cases it will be recognized that references to the "invention" will refer to one or more, but not necessarily all, of the subject matter described in the claims.
All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., "such as") provided with respect to certain embodiments herein, is intended merely to better illuminate the invention and does not pose a limitation on the scope of the invention claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the invention.
The various terms used herein are shown below. Where no term is defined below as used in the claims, the broadest definition persons in the pertinent art have given that term as reflected in printed publications and issued patents at the time of filing.
Example 1
In this embodiment, as shown in fig. 1, the present invention provides a control method for an intelligent loading and docking line based on the internet of things and machine vision, which includes the following steps:
s1, detecting vehicle position information and vehicle identity information of a transport vehicle entering a designated area of a port loading line, and executing S2 when the vehicle position information and the vehicle identity information meet preset conditions;
s2, identifying the action state of a hopper and the residual state of hopper materials of the transport vehicle in a visual detection mode, and then executing S3;
and S3, judging whether the transport vehicle is allowed to leave the designated area of the port loading refute line or not according to the hopper action state and the hopper material residual state of the transport vehicle.
Specifically, the preset conditions described in S1 are: the preset condition of the vehicle position information is that the transport vehicle is positioned in a designated area of the port loading line and is detected; the preset condition of the vehicle identity information is that the license plate identity of the transport vehicle is in the range of a specified license plate or a specified license plate group.
Specifically, in S1, the vehicle identification information of the transport vehicle is identified by means of OCR license plate recognition.
Optimally in this embodiment, between S1 and S2, the control method further comprises the steps of:
s102, judging whether the transport vehicle is in a heavy weight state, if so, executing S2, and if not, sending out prompt information; specifically, the prompt message: the transport vehicle is not weighed and is unloaded after weighing.
In the embodiment, in S1, the detection of the vehicle position information is performed on the transport vehicle entering the designated area of the port loading line by means of laser radar detection.
In the embodiment, in S2, the residual state of the hopper material is detected by using a deep convolutional neural network based on deep learning.
Specifically, the deep learning-based method for applying the deep convolutional neural network includes the following steps (1) - (4):
step (1), pretreatment is carried out; obtaining pictures from a video stream, cutting the pictures to remove redundant background, performing Gaussian blur noise reduction, defogging, dust removal and other image algorithm processing on the pictures, reducing the influence of severe environment and enhancing the image quality;
step (2), training is carried out; the detection network is improved by taking a deep learning convolutional neural network YOLOV5 as a main body: the dust blur pictures with different degrees are input to realize data enhancement; the Focus layer is changed into a single convolution layer, so that training and reasoning are accelerated; the number of output layer channels is reduced, and the classification of excessive redundancy is reduced; the cosine learning rate is adjusted to be linear, and the detection result is improved;
step (3), detecting; the network is deployed on an Intel server, and converted into an Intel deep learning reasoning framework OPENVINO to accelerate network reasoning, a detection result is output, then the detection result with a low threshold value is removed through a non-maximum suppression algorithm, whether materials exist in the vehicle or not can be determined according to the retained result, whether the vehicle is in a bucket or not, whether personnel exist on an operation platform or not and the positions of the personnel exist in the operation platform or not; if no material is detected, a further secondary confirmation is made: detecting a material area by using an edge detection Canny algorithm, converting the material area into a gray level image, and considering that the material exists in the hopper when the material area is larger than a threshold value;
step (4), obtaining a final result; and detecting a plurality of pictures, and sorting according to the times and the confidence coefficient of the detection results to select the most reliable detection result.
Optimally in the embodiment, in S3, when the transport vehicle completes the bucket lifting action and the bucket falling action and the hopper has no material residue, the transport vehicle is allowed to drive away from a designated area of a port loading line (specific equipment is implemented as follows: corresponding barrier lift, corresponding LED device prompts "operation is completed to slow down, stock farm PC end prompts" normal ", traffic light device displays green light); in S3, when the transport vehicle completes the bucket lifting action but does not complete the bucket falling action, the transport vehicle is not allowed to drive away from the appointed area of the port loading refuting line (specific equipment execution is as follows: corresponding barrier gate is not lifted, corresponding LED device prompts that the operation is not completed, please return to the platform to complete the operation, the PC end of the warehouse prompts that the operation is abnormal prompt and alarm, the traffic light device displays red light and the buzzer alarms); in S3, when the transport vehicle does not complete the bucket lifting action or the bucket falling action, the transport vehicle is not allowed to drive away from the appointed area of the port loading refute line (specific equipment execution is as follows, corresponding barrier gate is not lifted, corresponding LED device prompts that the operation is not completed, please return to a platform to complete the operation, a PC end of a warehouse prompts that the operation is abnormal, a traffic light device displays a red light, and a buzzer alarms); in S3, when the transport vehicle completes the bucket lifting action and the bucket falling action and the hopper has material residues, the transport vehicle is not allowed to drive away from the appointed area of the port loading refuting line (specific equipment execution is as follows: corresponding barrier gate is not lifted, corresponding LED device prompts that the operation is not completed, please return to the platform to complete the operation, the PC end of the warehouse prompts that the operation is abnormal prompts and alarms, and the traffic light device displays red lights and a buzzer alarms).
Specifically, the specific business processing flow related to S3 is as follows (S10-S40):
s10, reversing the vehicle of the loading platform into the vehicle, opening a barrier gate at the entrance of the loading platform, detecting that the transportation vehicle enters, and closing the barrier gate to prevent other vehicles from entering;
s20, a transport vehicle enters a loading and unloading position of a loading and unloading platform, an IO signal of a laser radar sensor is triggered, an intelligent terminal receives the trigger signal, the intelligent terminal informs license plate recognition to begin recognition, vehicle information is acquired, vehicle verification is carried out after the vehicle information is acquired, the vehicle is verified, verification passes, voice can be passed, an LED reminds a driver of starting operation, if the verification does not pass, voice, an LED and a buzzer alarm remind the driver of contacting a service person, and after related abnormality is processed through interphone communication, the driver can operate;
s30, after verification, entering an operation state, processing machine vision identification data, entering a discharge state when a vehicle is visually identified to lift a hopper, informing to enter a discharge completion state when no residue is identified in the vehicle hopper, after discharge is completed, detecting that the hopper is restored to a falling state, marking as the operation completion, prompting a driver of the completion of the operation through an LED, voice and the like, and operating the lifting of a barrier gate to release the vehicle at the moment; if the completion is not detected or the unloading is not completed, the brake is not lifted, and a driver is also prompted to return to the operation platform to complete related operation;
and S40, after the vehicle leaves, all the equipment is restored to the initial state, and the next working vehicle is waited for.
Optimally in the embodiment, in S3, when the transport vehicle is not allowed to drive away from the appointed area of the port loading line, an alarm prompt is sent;
optimally in the embodiment, when the transport vehicle is not allowed to drive away from the appointed area of the port loading refute line, the gate equipment in the port loading refute line is controlled not to be opened;
in S1, when any one of the vehicle position information and the vehicle identity information does not meet the preset condition, an alarm prompt is sent; specifically, the alarm prompting flow is as follows: prompting the vehicle to have no operation instruction, contacting manual processing and/or sending an alarm signal; specifically, the vehicle identification information may be license plate information.
Specifically, the barrier gate device in this embodiment may also use traffic lights instead of the same, or be a barrier gate device with traffic lights.
Optimally in this embodiment, before S1, the method further comprises the following steps:
s100, controlling the barrier equipment in the port loading refute line to open a barrier, and allowing the transport vehicle to drive to the appointed area of the port loading refute line through the barrier equipment.
Specifically, before S100, data access of the internet of things device is further required; for example, the intelligent terminal M2 of the Internet of things and the intelligent terminal M3 of the Internet of things are deployed, the intelligent terminal M2 of the Internet of things is connected with a laser radar and a barrier gate, data interaction is carried out with the intelligent terminal M2 of the Internet of things, an LED and a cloud platform through a network, a network relay receives a trigger IO signal of a laser radar sensor, and data of machine vision and license plate recognition are interacted with the intelligent terminal M3 of the Internet of things through the network; after the equipment data is accessed, the data analysis of the Internet of things equipment is carried out, a specific intelligent terminal receives a visual identification signal, a laser radar signal and a license plate identification signal in real time, and the switch of the barrier gate, the change of the indicator light and the voice are controlled by processing the received information to guide a vehicle driver to operate.
Example 2
In this embodiment, as shown in fig. 2, the invention provides a control system of an intelligent loading line based on the internet of things and machine vision, which comprises a port operation management platform, a deep learning model, a vision device and a vehicle information identification subsystem;
the deep learning model is embedded in a port operation management platform which is respectively in communication connection with the vision equipment and the vehicle information identification subsystem;
the port operation management platform is used for managing, monitoring and analyzing port loading lines and transport vehicles; the port operation management platform judges whether the transport vehicle is allowed to leave a designated area of a port loading line or not according to the action state of a hopper of the transport vehicle and the residual state of hopper materials;
the deep learning model is used for analyzing the residual state of the hopper materials by using a deep convolutional neural network;
the visual equipment is used for visually detecting the action state of the hopper of the transport vehicle and the residual state of the hopper material;
the vehicle information identification subsystem is used for detecting vehicle position information and vehicle identity information of a transport vehicle entering a designated area of the port loading line.
Specifically, the port operation management platform also has the following functions:
1) Basic data management, supporting vehicle information input, and generating operation instructions of a loading and docking platform;
2) Remote monitoring, supporting a remote control loading platform, acquiring a vehicle state, checking operation details, and pushing abnormal operation prompts;
3) Recording and analyzing, recording data of each operation, and performing statistical analysis.
Preferably, the vehicle information recognition subsystem includes a vehicle position information recognition device and a vehicle identity information recognition device; the port operation management platform is respectively in communication connection with the vehicle position information identification equipment and the vehicle identity information identification equipment;
the vehicle position information identification device is used for detecting vehicle position information of a transport vehicle entering a designated area of the port loading line;
the vehicle identity information identification equipment is used for detecting vehicle identity information of a transport vehicle entering a designated area of the port loading line;
the port job management platform also performs the following control logic:
when the vehicle position information and the vehicle identity information meet preset conditions, the hopper action state and the hopper material residual state of the transport vehicle are identified in a visual detection mode.
Example 3
In this embodiment, as shown in fig. 3, the present invention proposes a control system for an intelligent loading line based on the internet of things and machine vision, including a port operation management platform, a deep learning model, a vision device and a vehicle information recognition subsystem;
the deep learning model is arranged on an independent intelligent analysis platform, the independent intelligent analysis platform performs information interaction with a port operation management platform, the independent intelligent analysis platform is in communication connection with the vision equipment, and the port operation management platform is in communication connection with the vehicle information identification subsystem;
the port operation management platform is used for managing, monitoring and analyzing port loading lines and transport vehicles; the port operation management platform judges whether the transport vehicle is allowed to leave a designated area of a port loading line or not according to the action state of a hopper of the transport vehicle and the residual state of hopper materials;
the deep learning model is used for analyzing the residual state of the hopper materials by using a deep convolutional neural network;
the visual equipment is used for visually detecting the action state of the hopper of the transport vehicle and the residual state of the hopper material;
the vehicle information identification subsystem is used for detecting vehicle position information and vehicle identity information of a transport vehicle entering a designated area of the port loading line.
Preferably, the vehicle information recognition subsystem includes a vehicle position information recognition device and a vehicle identity information recognition device; the port operation management platform is respectively in communication connection with the vehicle position information identification equipment and the vehicle identity information identification equipment;
the vehicle position information identification device is used for detecting vehicle position information of a transport vehicle entering a designated area of the port loading line; specifically, the vehicle position information identification device is a laser radar sensor with a detection range in a designated area of a port loading line;
the vehicle identity information identification equipment is used for detecting vehicle identity information of a transport vehicle entering a designated area of the port loading line; specifically, the vehicle identity information identifying device may be a camera device or an RFID identifying device;
the port job management platform also performs the following control logic:
when the vehicle position information and the vehicle identity information meet preset conditions, the hopper action state and the hopper material residual state of the transport vehicle are identified in a visual detection mode.
Example 4
Example 4 is a further improvement over example 1;
in this embodiment, the present invention still further proposes a computer-readable storage medium storing a program code for execution, the program code comprising instructions for implementing the method according to any one of embodiment 1.
The foregoing examples illustrate only a few embodiments of the invention and are described in detail herein without thereby limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.
Claims (10)
1. The intelligent loading and docking line control method based on the Internet of things and machine vision is characterized by comprising the following steps of:
s1, detecting vehicle position information and vehicle identity information of a transport vehicle entering a designated area of a port loading line, and executing S2 when the vehicle position information and the vehicle identity information meet preset conditions;
s2, identifying the action state of a hopper and the residual state of hopper materials of the transport vehicle in a visual detection mode, and then executing S3;
and S3, judging whether the transport vehicle is allowed to leave the designated area of the port loading refute line or not according to the hopper action state and the hopper material residual state of the transport vehicle.
2. The control method of the intelligent loading line based on the internet of things and the machine vision according to claim 1, wherein between S1 and S2, the control method further comprises the following steps:
s102, judging whether the transport vehicle is in a heavy weight state, if so, executing S2, and if not, sending out prompt information.
3. The control method of the intelligent loading and plugging line based on the internet of things and machine vision according to claim 1, wherein in S1, vehicle position information of a transport vehicle entering a designated area of the port loading and plugging line is detected by a laser radar detection method.
4. The control method of the intelligent loading line based on the internet of things and the machine vision according to claim 1, wherein in S2, the residual state of the hopper material is detected by applying a deep convolutional neural network based on deep learning.
5. The control method of intelligent loading and plugging line based on internet of things and machine vision according to claim 1, wherein in S3, when the transport vehicle completes the bucket lifting and dropping actions and the hopper has no material residue, the transport vehicle is allowed to leave the designated area of the port loading and plugging line; in S3, when the transport vehicle finishes the bucket lifting action but does not finish the bucket falling action, the transport vehicle is not allowed to leave the appointed area of the port loading refute line; in S3, when the transport vehicle does not complete the bucket lifting action or the bucket falling action, the transport vehicle is not allowed to leave the appointed area of the port loading refute line; in S3, when the transport vehicle completes the bucket lifting and dropping actions and the hopper has material residues, the transport vehicle is not allowed to leave the designated area of the port loading line.
6. The control method of an intelligent loading and plugging line based on the internet of things and machine vision according to any one of claims 1 to 5, wherein in S3, when the transport vehicle is not allowed to drive away from the designated area of the port loading and plugging line, an alarm prompt is sent;
in S3, when the transport vehicle is not allowed to drive away from the appointed area of the port loading refute line, controlling the barrier gate equipment in the port loading refute line not to open the gate;
in S1, when any one of the vehicle position information and the vehicle identity information does not satisfy the preset condition, an alarm prompt is sent.
7. The control method of the intelligent loading line based on the internet of things and the machine vision according to any one of claims 1 to 5, further comprising the following steps before S1:
s100, controlling the barrier equipment in the port loading refute line to open a barrier, and allowing the transport vehicle to drive to the appointed area of the port loading refute line through the barrier equipment.
8. The intelligent refuting line control system based on the Internet of things and the machine vision is characterized by comprising a port operation management platform, a deep learning model, visual equipment and a vehicle information identification subsystem;
if the deep learning model is embedded in the port operation management platform, the port operation management platform is respectively in communication connection with the vision equipment and the vehicle information identification subsystem;
if the deep learning model is arranged on an independent intelligent analysis platform, the independent intelligent analysis platform performs information interaction with a port operation management platform, the independent intelligent analysis platform is in communication connection with visual equipment, and the port operation management platform is in communication connection with a vehicle information identification subsystem;
the port operation management platform is used for managing, monitoring and analyzing port loading lines and transport vehicles; the port operation management platform judges whether the transport vehicle is allowed to leave a designated area of a port loading line or not according to the action state of a hopper of the transport vehicle and the residual state of hopper materials;
the deep learning model is used for analyzing the residual state of the hopper materials by using a deep convolutional neural network;
the visual equipment is used for visually detecting the action state of the hopper of the transport vehicle and the residual state of the hopper material;
the vehicle information identification subsystem is used for detecting vehicle position information and vehicle identity information of a transport vehicle entering a designated area of the port loading line.
9. The control system of the intelligent loading line based on the internet of things and the machine vision according to claim 8, wherein the vehicle information identification subsystem comprises a vehicle position information identification device and a vehicle identity information identification device; the port operation management platform is respectively in communication connection with the vehicle position information identification equipment and the vehicle identity information identification equipment;
the vehicle position information identification device is used for detecting vehicle position information of a transport vehicle entering a designated area of the port loading line;
the vehicle identity information identification equipment is used for detecting vehicle identity information of a transport vehicle entering a designated area of the port loading line;
the port job management platform also performs the following control logic:
when the vehicle position information and the vehicle identity information meet preset conditions, the hopper action state and the hopper material residual state of the transport vehicle are identified in a visual detection mode.
10. A computer readable storage medium, characterized in that the computer readable storage medium stores a program code for execution, the program code comprising instructions for implementing the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311662983.0A CN117764468A (en) | 2023-12-06 | 2023-12-06 | Intelligent loading and plugging line control method and system based on Internet of things and machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311662983.0A CN117764468A (en) | 2023-12-06 | 2023-12-06 | Intelligent loading and plugging line control method and system based on Internet of things and machine vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117764468A true CN117764468A (en) | 2024-03-26 |
Family
ID=90315539
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311662983.0A Pending CN117764468A (en) | 2023-12-06 | 2023-12-06 | Intelligent loading and plugging line control method and system based on Internet of things and machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117764468A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107390689A (en) * | 2017-07-21 | 2017-11-24 | 北京图森未来科技有限公司 | Realize system and method, the relevant device of vehicle automatic transportation |
| CN108154334A (en) * | 2018-01-22 | 2018-06-12 | 广州玖峰信息科技有限公司 | A kind of Bulk Grain automobile unloading and into export operation dynamic and intelligent managing and control system |
| KR20180137978A (en) * | 2017-06-20 | 2018-12-28 | 인제대학교 산학협력단 | Unattended Port Entrance and Exit Gate Control of Container Truck Providing Efficiency and Security |
| KR102206658B1 (en) * | 2020-08-14 | 2021-01-22 | 아이티플래닛 주식회사 | Vision camera system to be communicated with Terminal Operating System through middleware in a port container terminal and method thereof |
| CN112978414A (en) * | 2019-12-12 | 2021-06-18 | 上海梅山钢铁股份有限公司 | Automatic truck of blowing awards hopper |
| CN114394445A (en) * | 2022-01-13 | 2022-04-26 | 青岛杰瑞工控技术有限公司 | Port granary intelligent loading and unloading system based on industrial internet |
| CN115496757A (en) * | 2022-11-17 | 2022-12-20 | 山东新普锐智能科技有限公司 | Hydraulic plate turnover surplus material amount detection method and system based on machine vision |
| CN115937491A (en) * | 2022-10-09 | 2023-04-07 | 华能国际电力股份有限公司上海石洞口第二电厂 | Method and system for identifying bulk coal in hopper of ship unloader based on computer vision |
| CN116092285A (en) * | 2022-11-28 | 2023-05-09 | 广州港集团有限公司 | Intelligent port dispatching command system and method for dealing with container trucks |
| CN116540739A (en) * | 2017-07-21 | 2023-08-04 | 北京图森智途科技有限公司 | Method and system for realizing automatic loading and unloading of vehicle and related equipment |
-
2023
- 2023-12-06 CN CN202311662983.0A patent/CN117764468A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20180137978A (en) * | 2017-06-20 | 2018-12-28 | 인제대학교 산학협력단 | Unattended Port Entrance and Exit Gate Control of Container Truck Providing Efficiency and Security |
| CN107390689A (en) * | 2017-07-21 | 2017-11-24 | 北京图森未来科技有限公司 | Realize system and method, the relevant device of vehicle automatic transportation |
| CN116540739A (en) * | 2017-07-21 | 2023-08-04 | 北京图森智途科技有限公司 | Method and system for realizing automatic loading and unloading of vehicle and related equipment |
| CN108154334A (en) * | 2018-01-22 | 2018-06-12 | 广州玖峰信息科技有限公司 | A kind of Bulk Grain automobile unloading and into export operation dynamic and intelligent managing and control system |
| CN112978414A (en) * | 2019-12-12 | 2021-06-18 | 上海梅山钢铁股份有限公司 | Automatic truck of blowing awards hopper |
| KR102206658B1 (en) * | 2020-08-14 | 2021-01-22 | 아이티플래닛 주식회사 | Vision camera system to be communicated with Terminal Operating System through middleware in a port container terminal and method thereof |
| CN114394445A (en) * | 2022-01-13 | 2022-04-26 | 青岛杰瑞工控技术有限公司 | Port granary intelligent loading and unloading system based on industrial internet |
| CN115937491A (en) * | 2022-10-09 | 2023-04-07 | 华能国际电力股份有限公司上海石洞口第二电厂 | Method and system for identifying bulk coal in hopper of ship unloader based on computer vision |
| CN115496757A (en) * | 2022-11-17 | 2022-12-20 | 山东新普锐智能科技有限公司 | Hydraulic plate turnover surplus material amount detection method and system based on machine vision |
| CN116092285A (en) * | 2022-11-28 | 2023-05-09 | 广州港集团有限公司 | Intelligent port dispatching command system and method for dealing with container trucks |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11145043B2 (en) | Using unmanned aerial vehicles to inspect autonomous vehicles | |
| US8181868B2 (en) | System, method, apparatus, and computer program product for monitoring the transfer of cargo to and from a transporter | |
| JP2005522390A (en) | Method and apparatus for automatic optical recognition of container codes using wharf container cranes, including position identification | |
| US20230161351A1 (en) | System for monitoring inventory of a warehouse or yard | |
| CN116208745A (en) | Dock ship berthing monitoring and recording system and method | |
| WO2022257397A1 (en) | Intelligent storage yard management system and method | |
| WO2021258195A1 (en) | Image-based system and method for shipping container management with edge computing | |
| CN112258038A (en) | Method, device and equipment for identifying platform use state and vehicle loading and unloading state | |
| CN117764468A (en) | Intelligent loading and plugging line control method and system based on Internet of things and machine vision | |
| US20230114688A1 (en) | Edge computing device and system for vehicle, container, railcar, trailer, and driver verification | |
| CN112422672A (en) | Electronic escorting system and escorting method | |
| US20230112116A1 (en) | Container Load Performance Metric Aggregation | |
| CN111612111B (en) | Mobile inspection monitoring device | |
| CN116306997A (en) | Platform dispatching system | |
| CN115557411A (en) | Monitoring device for crawler crane | |
| CN115359416A (en) | Intelligent early warning system for railway freight yard sky eye | |
| CN114701934A (en) | Security control method, device and system for elevator, cloud platform and storage medium | |
| EP3810501B1 (en) | Ship cargo ramp positioning system | |
| CN113642509A (en) | Garbage bin overflow state detection method and device, storage medium and electronic equipment | |
| CN111812610A (en) | Overwater target supervision system and method, terminal device and storage medium | |
| EP4241928A1 (en) | Method for operating a material handling apparatus | |
| KR20240076086A (en) | System for assuring safety around container trailer in port and Method thereof | |
| US20240078499A1 (en) | System for monitoring transportation, logistics, and distribution facilities | |
| KR102447794B1 (en) | Automatic Deceleration Braking System with the Preventing Collision and Narrowly Stuck Accident of Moving Equipment for Harbor | |
| US20240071046A1 (en) | System and Method for Load Bay State Detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |