CN118131699A - Industrial equipment, control method and device thereof, readable storage medium and program product - Google Patents
Industrial equipment, control method and device thereof, readable storage medium and program product Download PDFInfo
- Publication number
- CN118131699A CN118131699A CN202410092021.4A CN202410092021A CN118131699A CN 118131699 A CN118131699 A CN 118131699A CN 202410092021 A CN202410092021 A CN 202410092021A CN 118131699 A CN118131699 A CN 118131699A
- Authority
- CN
- China
- Prior art keywords
- image
- images
- industrial
- industrial equipment
- panoramic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000012545 processing Methods 0.000 claims abstract description 86
- 238000013507 mapping Methods 0.000 claims description 34
- 238000002372 labelling Methods 0.000 claims description 13
- 239000011159 matrix material Substances 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 4
- 230000004927 fusion Effects 0.000 claims description 4
- 239000003245 coal Substances 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 238000005520 cutting process Methods 0.000 description 8
- 238000004422 calculation algorithm Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 238000005265 energy consumption Methods 0.000 description 4
- 238000009412 basement excavation Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000005065 mining Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000005641 tunneling Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 229910052500 inorganic mineral Inorganic materials 0.000 description 1
- 239000011707 mineral Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Image Processing (AREA)
Abstract
The invention relates to the technical field of equipment control, and provides industrial equipment, a control method, a control device, a readable storage medium and a program product thereof. The control method of the industrial equipment comprises the following steps: acquiring N first images of a workplace where industrial equipment is located, wherein N is a positive integer greater than or equal to 2; performing image stitching processing on the N first images to obtain panoramic images of the workplace; performing image recognition processing on the panoramic image, and determining the running track of industrial equipment; and controlling the industrial equipment to work according to the running track.
Description
Technical Field
The present invention relates to the field of device control technology, and in particular, to an industrial device, a control method, an apparatus, a readable storage medium, and a program product thereof.
Background
Along with the development of science and technology, people's life is changed over the sky and over the earth, and energy mineral resources (such as coal mines) are used as non-renewable energy sources, so that the application is very wide, and most of existing coal mining uses industrial equipment such as heading machines for tunneling work. However, the density of the personnel on the tunneling working face is high, the environment is severe, the geology is complex, the accident risks such as gas outburst exist, and the personnel safety of underground personnel is seriously endangered. It can be seen that the intelligentization and the less humanization of the tunnel excavation are urgent demands of coal mines. However, in severe environments such as underground roadways, the control difficulty of industrial equipment such as heading machines is high, and the control accuracy is low, so that the control method for the industrial equipment in the present stage has the problems of low control accuracy, low control efficiency and the like.
Disclosure of Invention
The invention aims to at least solve the technical problems of lower control precision, lower control efficiency and the like in the prior art or related technologies.
To this end, a first aspect of the invention consists in proposing a control method for an industrial plant.
A second aspect of the invention is directed to a control device for an industrial plant.
A third aspect of the present invention is directed to a control apparatus for an industrial plant.
A fourth aspect of the invention is directed to a readable storage medium.
A fifth aspect of the invention is directed to a computer program product.
A sixth aspect of the invention is directed to an industrial apparatus.
In view of this, according to a first aspect of the present invention, there is provided a control method of an industrial apparatus, comprising: acquiring N first images of a workplace where industrial equipment is located, wherein N is a positive integer greater than or equal to 2; performing image stitching processing on the N first images to obtain panoramic images of the workplace; performing image recognition processing on the panoramic image, and determining the running track of industrial equipment; and controlling the industrial equipment to work according to the running track.
According to the control method of the industrial equipment, the N first images are subjected to image stitching processing to obtain the panoramic image of the workplace, the panoramic image is subjected to image recognition processing to determine the running track of the industrial equipment, the industrial equipment is controlled to work according to the running track, the accuracy of controlling the industrial equipment is improved, meanwhile, the efficiency of controlling the industrial equipment is improved, the industrial equipment can be ensured to run safely and accurately, accidents of the industrial equipment are avoided, and the energy consumption of the industrial equipment is reduced.
According to a second aspect of the present invention, there is provided a control apparatus of an industrial device, comprising: the processing module is used for acquiring N first images of a workplace where the industrial equipment is located, wherein N is a positive integer greater than or equal to 2; the processing module is also used for carrying out image stitching processing on the N first images so as to obtain panoramic images of the workplace; the processing module is also used for carrying out image recognition processing on the panoramic image and determining the running track of the industrial equipment; and the processing module is also used for controlling the industrial equipment to work according to the running track.
According to the technical scheme, the control device of the industrial equipment performs image stitching processing on the N first images to obtain a panoramic image of a workplace, performs image recognition processing on the panoramic image, determines the running track of the industrial equipment, controls the industrial equipment to work according to the running track, improves the accuracy of controlling the industrial equipment, improves the efficiency of controlling the industrial equipment, ensures that the industrial equipment can run safely and accurately, avoids accidents, and reduces the energy consumption of the industrial equipment.
According to a third aspect of the present invention, there is provided a control apparatus for an industrial device, comprising a processor and a memory, the memory storing a program or instructions which, when executed by the processor, implement the steps of the control method for an industrial device as in any of the above-mentioned aspects. Therefore, the control device of the industrial equipment has all the advantages of the control method of the industrial equipment in any of the above-mentioned technical aspects, and will not be described herein.
According to a fourth aspect of the present invention, there is provided a readable storage medium having stored thereon a program or instructions which, when executed by a processor, implement a control method of an industrial device as in any of the above-mentioned aspects. Therefore, the readable storage medium has all the advantages of the control method of the industrial equipment in any of the above-mentioned technical solutions, and will not be described herein.
According to a fifth aspect of the present invention, a computer program product is presented, comprising computer instructions which, when executed by a processor, implement a method of controlling an industrial device according to any of the above-mentioned aspects. Therefore, the readable storage medium has all the advantages of the control method of the industrial equipment in any of the above-mentioned technical solutions, and will not be described herein.
According to a sixth aspect of the present invention, there is provided an industrial apparatus comprising: the control device of the industrial equipment as defined in the above second aspect, or the control device of the industrial equipment as defined in the above third aspect, and/or the readable storage medium as defined in the above fourth aspect, thus has all the advantageous technical effects of the control device of the industrial equipment as defined in the above second aspect, or the control device of the industrial equipment as defined in the above third aspect, and/or the readable storage medium as defined in the above fourth aspect, which are not repeated here.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, or may be learned by practice of the invention.
Drawings
The foregoing and/or additional aspects and advantages of the invention will become apparent and may be better understood from the following description of embodiments taken in conjunction with the accompanying drawings in which:
FIG. 1 shows one of the flow diagrams of a control method of an industrial device in an embodiment of the invention;
FIG. 2 shows a second flow diagram of a control method of an industrial device in an embodiment of the invention;
FIG. 3 shows a third flow chart of a method of controlling an industrial plant in an embodiment of the invention;
FIG. 4 shows a fourth flow diagram of a control method of an industrial plant in an embodiment of the invention;
FIG. 5 shows a fifth flow diagram of a control method of an industrial plant in an embodiment of the invention;
FIG. 6 shows a sixth flow diagram of a control method of an industrial plant in an embodiment of the invention;
FIG. 7 shows a seventh flow diagram of a control method of an industrial device in an embodiment of the invention;
FIG. 8 shows one of the block diagrams of the control device of the industrial equipment in the embodiment of the present invention;
FIG. 9 shows a schematic diagram of a control device of an industrial plant in an embodiment of the invention;
Fig. 10 shows a second block diagram of the control apparatus of the industrial equipment in the embodiment of the present invention.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will be more clearly understood, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description. It should be noted that, without conflict, the embodiments of the present invention and features in the embodiments may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways than those described herein, and therefore the scope of the present invention is not limited to the specific embodiments disclosed below.
The industrial equipment, the control method, the device, the readable storage medium and the program product thereof according to the embodiments of the present application are described in detail below with reference to fig. 1 to 10 by means of specific embodiments and application scenarios thereof.
The execution subject of the technical scheme of the control method of the industrial equipment provided by the invention can be a control device, and can be determined according to actual use requirements, and is not particularly limited herein. In order to more clearly describe the control method of the industrial equipment provided by the invention, the following description will be made with the control device as the execution subject.
As shown in fig. 1, an embodiment of the present invention provides a control method of an industrial device, including:
102, acquiring N first images of a workplace where industrial equipment is located, wherein N is a positive integer greater than or equal to 2;
104, performing image stitching processing on the N first images to obtain panoramic images of the workplace;
step 106, performing image recognition processing on the panoramic image, and determining the running track of industrial equipment;
And 108, controlling the industrial equipment to work according to the running track.
In this embodiment, a control method of an industrial device is provided, where a control apparatus acquires N first images of a place where the industrial device is located, where N is a positive integer greater than or equal to 2, the industrial device is a device for industrial engineering, the first images are two-dimensional images of the place where the industrial device is located, and the place where the industrial device is located is a place where the industrial device is located.
By way of example, the industrial plant may be embodied as a smart heading machine.
By way of example, the workplace may be embodied as an underground coal mine.
The first image may be, for example, a real-time image of the underground coal mine.
And the control device performs image stitching processing on the N first images to obtain panoramic images of the workplace, wherein the panoramic images are panoramic images of the workplace.
By way of example, the panoramic image may be embodied as a 360 degree panoramic image of an underground coal mine.
The control device performs image recognition processing on the panoramic image, determines the running track of the industrial equipment, and controls the industrial equipment to work according to the running track, wherein the running track is the running track of the industrial equipment.
The movement path may be embodied, for example, as a movement path of the heading machine.
The movement track may be embodied as an excavation track of a heading machine, for example.
It should be noted that, workplaces may be scenes such as underground tunnels, and industrial equipment such as heading machines have problems of high control difficulty and poor control accuracy due to bad conditions of the underground tunnels, and meanwhile, in environments such as the underground tunnels, great potential safety hazards exist for workers. The control method of the industrial equipment in the embodiment can reduce the number of workers on site, improve the accuracy of controlling the industrial equipment, avoid the industrial equipment from colliding with the workers, and ensure the safety of the workers.
In addition, the technical field of coal mining and the technical field of heading machines at the present stage do not relate to the technical scheme of splicing the N first images into panoramic images, but the control method of the industrial equipment in the embodiment can perform image splicing processing on the N first images to obtain panoramic images of a workplace, perform image recognition processing on the panoramic images, determine the running track of the industrial equipment, and control the industrial equipment to work according to the running track.
According to the control method of the industrial equipment, the panoramic image of a workplace is obtained by carrying out image stitching processing on the N first images, the panoramic image is subjected to image recognition processing, the running track of the industrial equipment is determined, the industrial equipment is controlled to work according to the running track, the accuracy of controlling the industrial equipment is improved, meanwhile, the efficiency of controlling the industrial equipment is improved, the industrial equipment can be ensured to run safely and accurately, accidents of the industrial equipment are avoided, and the energy consumption of the industrial equipment is reduced.
In some embodiments, optionally, as shown in fig. 2, a control method of an industrial device is provided, and image stitching processing is performed on N first images to obtain a panoramic image of a workplace, where the method includes:
Step 202, performing feature extraction processing on N first images to obtain N image feature data;
step 204, determining an image mapping relation according to the N image feature data;
and 206, stitching the N first images into a panoramic image based on the image mapping relation.
In this embodiment, the control device performs feature extraction processing on N first images, and determines N image feature data, where the N image feature data corresponds to the N first images one by one, and the image feature data is feature data in the first images.
The image feature data may be texture feature data in the first image, for example.
The image characteristic data may be, for example, color characteristic data in the first image.
The image feature data may be, for example, shape feature data in the first image.
The image feature data may be, for example, contour feature data in the first image.
The image characteristic data may be, for example, spatial relationship characteristic data in the first image. The control device determines an image mapping relation according to the N image characteristic data, wherein the image mapping relation is used for representing the splicing relation of any two first images in the N first images.
The image mapping relationship may be a stitching relationship of the N first images, for example.
And the control device splices the N first images into a panoramic image according to the image mapping relation.
Illustratively, the control device linearly fuses the N first images into a panoramic image according to the image mapping relationship.
According to the control method of the industrial equipment, according to the N image feature data, the image mapping relation of the N first images is determined, and then according to the image mapping relation, the N first images are linearly fused into the panoramic image, so that the image accuracy of the panoramic image is ensured, and the accuracy of the running track is further ensured.
In some embodiments, optionally, as shown in fig. 3, a control method of an industrial device is provided, where feature extraction processing is performed on N first images to obtain N image feature data, including:
Step 302, performing feature recognition processing on the N first images to obtain N feature pixel sets in the N first images;
step 304, data corresponding to the N feature pixel sets are obtained, so as to obtain N image feature data.
In this embodiment, the control device performs feature recognition processing on the N first images to obtain N feature pixel sets in the N first images, where the N feature pixel sets are in one-to-one correspondence with the N first images, and the feature pixel sets are a set of feature pixel points.
For example, the feature pixel set may comprise texture feature pixel points in the first image.
For example, the feature pixel set may include color feature pixel points in the first image.
For example, the feature pixel set may include shape feature pixel points in the first image.
For example, the feature pixel set may comprise spatially feature pixel points in the first image.
The control device acquires data corresponding to the N characteristic pixel sets to obtain N image characteristic data.
The control method of the industrial equipment in the embodiment performs feature recognition processing on the N first images to obtain N feature pixel sets in the N first images, and then obtains data corresponding to the N feature pixel sets to obtain N image feature data, so that the data accuracy of the image feature data is ensured, and the splicing accuracy of panoramic images is further ensured.
In some embodiments, optionally, as shown in fig. 4, a control method of an industrial device is provided, where determining an image mapping relationship according to N image feature data includes:
Step 402, a data coordinate system is established based on N image characteristic data;
step 404, determining global homography matrixes corresponding to the N first images according to the data coordinate system;
Step 406, determining an image mapping relation between the N first images based on the global homography matrix.
In this embodiment, the control device establishes a data coordinate system according to the N image feature data, and determines global homography matrices corresponding to the N first images based on the data coordinate system, where the data coordinate system is a coordinate system corresponding to the N first images, and the global homography matrices are transformation relationships for representing planes where the N first images are located.
The global homography matrix may be a correspondence matrix of planes where the N first images are located.
The control device determines image mapping relations among N first images according to the global homography matrix.
For example, the control device may obtain the image mapping relationship according to a global homography matrix operation.
According to the control method of the industrial equipment, a data coordinate system is established according to the N image feature data, then the global homography matrix corresponding to the N first images is determined based on the data coordinate system, and then the image mapping relation among the N first images is determined according to the global homography matrix, so that the data accuracy of the image feature data is ensured, and further the splicing accuracy of panoramic images is ensured.
In some embodiments, optionally, as shown in fig. 5, a control method of an industrial device is provided, where the stitching N first images into a panoramic image based on an image mapping relationship includes:
step 502, performing grid labeling processing on the N first images to obtain N labeled first images;
And step 504, performing image fusion on the N marked first images based on the image mapping relation to obtain panoramic images.
In this embodiment, the control device performs grid labeling processing on the N first images to obtain N first images after labeling, and then fuses the N first images after labeling into a panoramic image according to an image mapping relationship.
Illustratively, an optimized meshing algorithm is used to mesh N first images, and all pixel points and a local homography matrix in the same mesh are deformed, so that a panoramic image is obtained through block calculation.
According to the control method of the industrial equipment, grid labeling processing is conducted on the N first images to obtain N first images after labeling, the N first images after labeling are fused into a panoramic image according to an image mapping relation, image accuracy of the panoramic image is guaranteed, and data accuracy of a running track is further guaranteed.
In some embodiments, optionally, as shown in fig. 6, a control method of an industrial device is provided, where the control method of the industrial device includes:
step 602, controlling N image acquisition devices to acquire images of a workplace so as to obtain N first images;
step 604, performing image stitching processing on the N first images to obtain a panoramic image of the workplace;
Step 606, performing image recognition processing on the panoramic image, and determining the running track of industrial equipment;
step 608, controlling the industrial equipment to work according to the running track.
In this embodiment, the industrial apparatus comprises N image capturing devices, wherein the capturing directions of the N image capturing devices are all different.
The image acquisition device may be embodied as a camera, for example.
The control device controls the N image acquisition devices to acquire images of the workplace so as to obtain N first images.
For example, the control device may control the N image capturing devices to perform video capturing on the workplace to obtain N video data.
According to the control method of the industrial equipment, the N image acquisition devices are controlled to acquire the images of the workplace, so that N first images are obtained, the image accuracy of the first images is guaranteed, and the image accuracy of the panoramic image is guaranteed.
In some embodiments, optionally, as shown in fig. 7, a control method of an industrial device is provided, performing image recognition processing on a panoramic image, and determining a running track of the industrial device, including:
step 702, performing image recognition processing on the panoramic image to obtain a working area image of industrial equipment;
Step 704, track planning is performed on the industrial equipment based on the working area image to obtain a running track.
In this embodiment, the control device performs image recognition processing on the panoramic image to obtain a working area image of the industrial equipment, wherein the working area image is image data of the working area of the industrial equipment.
The working area image may be an image of a cutting plane, for example.
And the control device performs track planning on the industrial equipment according to the working area image so as to obtain a running track.
Illustratively, texture recognition is performed on the work area image to determine the trajectory.
The control method of the industrial equipment in the embodiment carries out image recognition processing on the panoramic image to obtain a working area image of the industrial equipment, and then carries out track planning on the industrial equipment according to the working area image to obtain a running track, so that the accuracy of the running track is ensured, and the control precision of the industrial equipment is ensured.
As shown in fig. 8, in an embodiment of the present invention, there is provided a control apparatus 800 of an industrial device, including:
A processing module 802, configured to obtain N first images of a workplace where an industrial device is located, where N is a positive integer greater than or equal to 2;
The processing module 802 is further configured to perform image stitching processing on the N first images to obtain a panoramic image of the workplace;
the processing module 802 is further configured to perform image recognition processing on the panoramic image, and determine a running track of the industrial device;
the processing module 802 is further configured to control the industrial device to operate according to the running track.
In this embodiment, a control apparatus 800 of an industrial device is provided, where a processing module 802 acquires N first images of a place where the industrial device is located, where N is a positive integer greater than or equal to 2, the industrial device is a device for industrial engineering, the first images are two-dimensional images of the place where the industrial device is located, and the place where the industrial device is located is a place where the industrial device is located.
By way of example, the industrial plant may be embodied as a smart heading machine.
By way of example, the workplace may be embodied as an underground coal mine.
The first image may be, for example, a real-time image of the underground coal mine.
The processing module 802 performs image stitching processing on the N first images to obtain a panoramic image of the workplace, where the panoramic image is a panoramic image of the workplace.
By way of example, the panoramic image may be embodied as a 360 degree panoramic image of an underground coal mine.
The processing module 802 performs image recognition processing on the panoramic image, determines a running track of the industrial equipment, and controls the industrial equipment to work according to the running track, wherein the running track is a track of the industrial equipment running.
The movement track may be embodied as an excavation track of a heading machine, for example.
The movement path may be embodied, for example, as a movement path of the heading machine.
The control device 800 of the industrial equipment in this embodiment performs image stitching processing on the N first images to obtain a panoramic image of a workplace, performs image recognition processing on the panoramic image, determines a running track of the industrial equipment, and controls the industrial equipment to work according to the running track, so that accuracy in controlling the industrial equipment is improved, efficiency in controlling the industrial equipment is improved, safety and accuracy in running the industrial equipment are ensured, accidents of the industrial equipment are avoided, and energy consumption of the industrial equipment is reduced.
In some embodiments, optionally, the control device 800 of the industrial apparatus further comprises:
the processing module 802 is further configured to perform feature extraction processing on the N first images to obtain N image feature data, where the N image feature data corresponds to the N first images one to one;
the processing module 802 is further configured to determine an image mapping relationship according to the N image feature data, where the image mapping relationship is used to represent a stitching relationship of any two first images in the N first images;
the processing module 802 is further configured to stitch the N first images into a panoramic image based on the image mapping relationship.
The control device 800 of the industrial equipment in this embodiment determines the image mapping relationship of the N first images according to the N image feature data, and then linearly fuses the N first images into a panoramic image according to the image mapping relationship, so that the image accuracy of the panoramic image is ensured, and the accuracy of the running track is further ensured.
In some embodiments, optionally, the control device 800 of the industrial apparatus further comprises:
the processing module 802 is further configured to perform feature recognition processing on the N first images to obtain N feature pixel sets in the N first images, where the N feature pixel sets are in one-to-one correspondence with the N first images;
The processing module 802 is further configured to obtain data corresponding to the N feature pixel sets, so as to obtain N image feature data.
The control device 800 of the industrial equipment in this embodiment performs feature recognition processing on the N first images to obtain N feature pixel sets in the N first images, and then obtains data corresponding to the N feature pixel sets to obtain N image feature data, so that the data accuracy of the image feature data is ensured, and the stitching accuracy of the panoramic image is further ensured.
In some embodiments, optionally, the control device 800 of the industrial apparatus further comprises:
the processing module 802 is further configured to establish a data coordinate system based on the N image feature data;
the processing module 802 is further configured to determine global homography matrices corresponding to the N first images according to the data coordinate system;
The processing module 802 is further configured to determine an image mapping relationship between the N first images based on the global homography matrix.
The control device 800 of the industrial equipment in this embodiment establishes a data coordinate system according to the N image feature data, determines a global homography matrix corresponding to the N first images based on the data coordinate system, and further determines an image mapping relationship between the N first images according to the global homography matrix, thereby ensuring data accuracy of the image feature data and further ensuring stitching accuracy of panoramic images.
In some embodiments, optionally, the control device 800 of the industrial apparatus further comprises:
The processing module 802 is further configured to perform grid labeling processing on the N first images to obtain N labeled first images;
the processing module 802 is further configured to perform image fusion on the N first images after labeling based on the image mapping relationship, so as to obtain a panoramic image.
The control device 800 of the industrial equipment in this embodiment performs grid labeling processing on the N first images to obtain N first images after labeling, and fuses the N first images after labeling into a panoramic image according to an image mapping relationship, so that the image accuracy of the panoramic image is ensured, and the data accuracy of the running track is further ensured.
In some embodiments, optionally, the control device 800 of the industrial apparatus further comprises:
the processing module 802 is further configured to control the N image capturing devices to perform image capturing on the workplace, so as to obtain N first images.
The control device 800 of the industrial equipment in this embodiment performs image acquisition on the workplace by controlling the N image acquisition devices to obtain N first images, so that the image accuracy of the first images is ensured, and the image accuracy of the panoramic image is ensured.
In some embodiments, optionally, the control device 800 of the industrial apparatus further comprises:
The processing module 802 is further configured to perform image recognition processing on the panoramic image to obtain a working area image of the industrial device;
the processing module 802 is further configured to perform trajectory planning on the industrial device based on the working area image, so as to obtain a running trajectory.
Illustratively, as shown in fig. 9, a control device 800 of an industrial apparatus may implement the steps of:
S1: in order to ensure the splicing effect, cameras with the same brand and specification are required to be used in the video splicing process, and the same setting is realized on the focal length and other parameters of the cameras before video splicing is performed;
S2: dividing the obtained video image into areas, carrying out feature registration in a specific area by using a Scale Invariant Feature Transform (SIFT) algorithm, and purifying feature pairing by using a random sample consensus (RANSAC) algorithm;
S3: performing position alignment between each divided grid through a DLT (direct Linear transformation) algorithm, and finally performing linear image fusion to output a panoramic video image with spliced completion;
S4: transmitting the obtained video image to an F1 substation of the heading machine body for parameter analysis, and uploading the video image to an underground centralized control room and an underground centralized control center computer in a wireless WIFI and optical fiber communication mode;
S5: the underground centralized control room and the underground centralized control center are respectively provided with a remote control device of the heading machine, and the heading machine is remotely controlled to move forwards, backwards, cut, lift and the like by utilizing the obtained video information and a real-time cutting track picture displayed by an upper computer system to complete the cutting footage work of the whole working face;
S6: the heading machine can realize the super-undermining alarm and display the real-time cutting track in the process of remotely operating cutting.
SIFT in step S2 is a scale invariant feature transform algorithm.
RANSAC in step S2 is a random sample consensus algorithm.
DLT in step S3 is a direct linear transformation method.
The step S4 further includes: the video monitoring picture and the bidirectional voice system control can be displayed in real time.
The step S5 further includes: the wireless remote controller equipped with the heading machine is powered on and the dial knob is placed in a remote control state, so that the position and posture state of the upper computer are confirmed to be displayed in a column in the remote control state. And opening communication software of an operation desk of the well computer and starting the operation desk, so that the remote control device can be used for controlling the heading machine to carry out cutting operation.
The step S6 further includes: and displaying a two-dimensional picture of the cutting state of the roadway cutting section on the upper computer interface, and monitoring the numerical value of each oil cylinder displacement sensor.
The control device 800 of the industrial equipment in this embodiment performs image recognition processing on the panoramic image to obtain a working area image of the industrial equipment, and performs trajectory planning on the industrial equipment according to the working area image to obtain a running trajectory, so that accuracy of the running trajectory is ensured, and control accuracy of the industrial equipment is ensured.
In some embodiments, optionally, as shown in fig. 10, a control apparatus 1000 of an industrial device is provided, where the control apparatus 1000 of an industrial device includes a processor 1002 and a memory 1004, and a program or an instruction is stored in the memory 1004, where the program or the instruction is executed by the processor 1002 to implement the steps of the control method of an industrial device in any of the above claims. Accordingly, the control device 1000 of the industrial equipment has all the advantages of the control method of the industrial equipment according to any one of the above-mentioned aspects, and will not be described in detail herein.
In some embodiments, optionally, a readable storage medium is provided, on which a program or an instruction is stored, which when executed by a processor, implements the method for controlling an industrial device as in any of the embodiments described above, thereby having all the advantageous technical effects of the method for controlling an industrial device as in any of the embodiments described above.
Among them, readable storage media such as Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk, and the like.
In some embodiments, optionally, a computer program product is provided, comprising computer instructions which, when executed by a processor, implement a method of controlling an industrial device as in any of the embodiments described above, thereby having all the beneficial technical effects of the method of controlling an industrial device as in any of the embodiments described above.
In some embodiments, optionally, an industrial apparatus is provided, comprising: the control device of the industrial equipment in any of the above embodiments and/or the readable storage medium in any of the above embodiments, thus has all the beneficial technical effects of the control device of the industrial equipment in any of the above embodiments and/or the readable storage medium in any of the above embodiments, and will not be described in detail herein.
In some embodiments, optionally, the industrial device is any one of: heading machine, excavator or drill carriage.
It is to be understood that in the claims, specification and drawings of the present invention, the term "plurality" means two or more, and unless otherwise explicitly defined, the orientation or positional relationship indicated by the terms "upper", "lower", etc. are based on the orientation or positional relationship shown in the drawings, only for the convenience of describing the present invention and making the description process easier, and not for the purpose of indicating or implying that the apparatus or element in question must have the particular orientation described, be constructed and operated in the particular orientation, so that these descriptions should not be construed as limiting the present invention; the terms "connected," "mounted," "secured," and the like are to be construed broadly, and may be, for example, a fixed connection between a plurality of objects, a removable connection between a plurality of objects, or an integral connection; the objects may be directly connected to each other or indirectly connected to each other through an intermediate medium. The specific meaning of the terms in the present invention can be understood in detail from the above data by those of ordinary skill in the art.
In the claims, specification, and drawings of the present invention, the descriptions of terms "one embodiment," "some embodiments," "particular embodiments," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In the claims, specification and drawings of the present invention, the schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (12)
1. A control method of an industrial apparatus, characterized by comprising:
Acquiring N first images of a workplace where the industrial equipment is located, wherein N is a positive integer greater than or equal to 2;
performing image stitching processing on the N first images to obtain panoramic images of the workplace;
performing image recognition processing on the panoramic image, and determining the running track of the industrial equipment;
And controlling the industrial equipment to work according to the running track.
2. The method according to claim 1, wherein the performing image stitching processing on the N first images to obtain a panoramic image of the workplace comprises:
performing feature extraction processing on the N first images to obtain N image feature data, wherein the N image feature data are in one-to-one correspondence with the N first images;
determining an image mapping relation according to the N image characteristic data, wherein the image mapping relation is used for representing the splicing relation of any two first images in the N first images;
And splicing the N first images into a panoramic image based on the image mapping relation.
3. The method according to claim 2, wherein the performing feature extraction processing on the N first images to obtain N image feature data includes:
performing feature recognition processing on the N first images to obtain N feature pixel sets in the N first images, wherein the N feature pixel sets are in one-to-one correspondence with the N first images;
and acquiring data corresponding to the N characteristic pixel sets to obtain N image characteristic data.
4. The method of controlling an industrial apparatus according to claim 2, wherein determining an image mapping relationship from the N pieces of image feature data includes:
establishing a data coordinate system based on the N image characteristic data;
According to the data coordinate system, determining N global homography matrixes corresponding to the first images;
And determining the image mapping relation among N first images based on the global homography matrix.
5. The method of controlling an industrial apparatus according to claim 2, wherein the stitching N first images into a panoramic image based on the image mapping relationship, comprises:
Grid labeling processing is carried out on the N first images so as to obtain N labeled first images;
And carrying out image fusion on the N marked first images based on the image mapping relation to obtain the panoramic image.
6. The method for controlling an industrial apparatus according to claim 1, wherein the industrial apparatus includes N image capturing devices, the capturing directions of the N image capturing devices are different, the capturing N first images of a workplace where the industrial apparatus is located includes:
And controlling N image acquisition devices to acquire images of the workplace so as to obtain N first images.
7. The control method of an industrial apparatus according to any one of claims 1 to 6, wherein the performing image recognition processing on the panoramic image, determining a running track of the industrial apparatus, comprises:
performing image recognition processing on the panoramic image to obtain a working area image of the industrial equipment;
And carrying out track planning on the industrial equipment based on the working area image so as to obtain the running track.
8. A control device of an industrial apparatus, characterized in that the control device of an industrial apparatus comprises:
The processing module is used for acquiring N first images of the workplace where the industrial equipment is located, wherein N is a positive integer greater than or equal to 2;
the processing module is further used for performing image stitching processing on the N first images to obtain panoramic images of the workplace;
The processing module is also used for carrying out image recognition processing on the panoramic image and determining the running track of the industrial equipment;
The processing module is also used for controlling the industrial equipment to work according to the running track.
9. A control device for an industrial apparatus, comprising:
a processor;
a memory in which a program or instructions are stored, the processor, when executing the program or instructions in the memory, implementing the steps of the control method of an industrial device as claimed in any one of claims 1 to 7.
10. A readable storage medium, characterized in that it has stored thereon a program or instructions which, when executed by a processor, implement the steps of the control method of an industrial device according to any one of claims 1 to 7.
11. A computer program product comprising computer instructions which, when executed by a processor, implement the steps of the control method of an industrial device as claimed in any one of claims 1 to 7.
12. An industrial device, comprising:
The control device of an industrial apparatus according to claim 8 or 9; and/or
The readable storage medium of claim 10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410092021.4A CN118131699A (en) | 2024-01-23 | 2024-01-23 | Industrial equipment, control method and device thereof, readable storage medium and program product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410092021.4A CN118131699A (en) | 2024-01-23 | 2024-01-23 | Industrial equipment, control method and device thereof, readable storage medium and program product |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118131699A true CN118131699A (en) | 2024-06-04 |
Family
ID=91239425
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410092021.4A Pending CN118131699A (en) | 2024-01-23 | 2024-01-23 | Industrial equipment, control method and device thereof, readable storage medium and program product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118131699A (en) |
-
2024
- 2024-01-23 CN CN202410092021.4A patent/CN118131699A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111119887B (en) | Remote intervention AR inspection system of mine fully-mechanized mining working face under holographic technology | |
| Behzadan et al. | Augmented reality visualization: A review of civil infrastructure system applications | |
| CA2347569C (en) | Self-contained mapping and positioning system utilizing point cloud data | |
| CN103867205B (en) | A kind of development machine tele-control system and method | |
| CN108590657B (en) | Excavation control method and control system for underground roadway | |
| US20200149248A1 (en) | System and method for autonomous operation of heavy machinery | |
| CN113746936B (en) | VR and AR distributed cooperation fully-mechanized coal mining face intelligent monitoring system | |
| CN112734370A (en) | BIM-based project supervision information management method and system | |
| CN111199584B (en) | Target object positioning virtual-real fusion method and device | |
| CN113982690B (en) | Control method, device, system and tunneling, anchoring and protecting integrated machine based on anchoring operation | |
| CN213338437U (en) | Unmanned excavator and remote management monitoring system thereof | |
| CN111246181B (en) | Robot monitoring method, system, equipment and storage medium | |
| CN110430538B (en) | Tunnel production command cockpit | |
| CN116030226A (en) | Reality visualization method, system, equipment and medium for power distribution network operation | |
| CN115935637A (en) | GIS digital twin-based transformer substation visual operation and maintenance system | |
| CN111868656B (en) | Operation control system, operation control method, device, equipment and medium | |
| EP4130658A1 (en) | Guide display device and crane equipped with same | |
| CN118131699A (en) | Industrial equipment, control method and device thereof, readable storage medium and program product | |
| EP4242585A2 (en) | Surveying assistance system, information display terminal, surveying assistance method, and surveying assistance program | |
| CN113218365A (en) | Roadway measuring device and using method | |
| EP4112824A1 (en) | Measuring system, measuring method, and measuring program for determining the positions of the site resources at a construction site | |
| Kamat et al. | GPS and 3DOF tracking for georeferenced registration of construction graphics in outdoor augmented reality | |
| CN116091602A (en) | Digital twin interaction method and system for tunneling robot | |
| Hsieh et al. | On-site visual construction management system based on the integration of SLAM-based AR and BIM on a handheld device | |
| CN113390409A (en) | Method for realizing SLAM technology through robot whole-course autonomous exploration navigation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |