CN113177932A - Method and device for dividing and tracking working area - Google Patents
Method and device for dividing and tracking working area Download PDFInfo
- Publication number
- CN113177932A CN113177932A CN202110544127.XA CN202110544127A CN113177932A CN 113177932 A CN113177932 A CN 113177932A CN 202110544127 A CN202110544127 A CN 202110544127A CN 113177932 A CN113177932 A CN 113177932A
- Authority
- CN
- China
- Prior art keywords
- video image
- working area
- preset condition
- tracking
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30164—Workpiece; Machine component
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method and a device for dividing and tracking a working area, wherein the method comprises the following steps: acquiring a first video image for evaluating the operation quality of a welding head, a second video image for evaluating the operation quality of the welding head in real time, a first preset condition and a preset tracking condition, and performing initialization on an original tracking position; performing video frame extraction processing on the second video image to generate a third video image; performing frame detection on the third video image to generate a fourth video image; performing working area division processing on the fourth video image to generate a fifth video image with a plurality of working areas, and screening the plurality of working areas in the fifth video image according to a first preset condition to generate a working area meeting the first preset condition; and analyzing the working area which accords with the first preset condition according to the original tracking position in the first video image and the preset tracking condition to generate an instruction for adjusting the original tracking position.
Description
Technical Field
The invention relates to the technical field of welding head operation quality detection, in particular to a working area dividing and tracking method and device.
Background
At present, 18 railway departments exist in China, and 180 railway sections are all provided with a recorder or a camera for recording welding videos. All welding head welding records are handed over to the railway section, and the problems in the welding process are expected to be found by viewing the video, so that the clear responsibility of the accident is facilitated. Welding involves multiple processes, each with multiple item locations that need to be stuck. The method is mainly used for positioning key moments such as preheating starting and stopping moment, reaction starting and stopping moment, sedation moment, demoulding moment, mould pushing moment and the like in the device through a visual method.
The defects of the prior art are that the welding amount is large, a plurality of welding heads are arranged in 5 thousands, each group is provided with 2 cameras, the recording time is about 40-50 minutes, if the groups are manually checked, a large amount of manpower and time are consumed, the video amount is too large, the time and the manpower are consumed, the efficiency is too low, and the video is checked only when the welding heads have problems.
Disclosure of Invention
The invention aims to solve the technical problem of providing a method and a device for dividing and tracking a working area aiming at the defects of the prior art.
The technical scheme for solving the technical problems is as follows: a workspace partitioning and tracking method, comprising:
acquiring a first video image for evaluating the operation quality of a welding head, a second video image for evaluating the operation quality of the welding head in real time, a first preset condition and a preset tracking condition, wherein an original tracking position is preset in the first video image;
the original tracking position is subjected to the initialization;
performing video frame extraction processing on the second video image to generate a third video image;
performing frame detection on the third video image to generate a fourth video image;
performing working area division processing on the fourth video image to generate a fifth video image with a plurality of working areas, wherein the plurality of working areas are respectively provided with a mould component;
screening a plurality of working areas in the fifth video image according to a first preset condition to generate a working area meeting the first preset condition;
and analyzing the working area which accords with the first preset condition according to the original tracking position in the first video image and the preset tracking condition to generate an instruction for adjusting the original tracking position.
Further, the first preset condition is that: reserving a working area with the most three parts in a plurality of working areas;
the step of screening the plurality of working areas in the fifth video image according to the first preset condition to generate the working areas meeting the first preset condition includes:
acquiring the number of parts contained in each working area in the fifth video image; wherein the parts comprise a mold part and a non-mold part;
sorting the working areas according to the number of the parts contained in each working area to form a working area list with the number of the parts gradually reduced;
and screening three working areas with the top rank in the working area list to generate the working areas meeting the first preset condition.
Further, the step of performing work area division processing on the fourth video image to generate a fifth video image having a plurality of work areas includes:
acquiring a second preset condition;
screening the mold parts and non-mold parts other than the mold parts in the fourth video image;
creating a working area centered on the mold part;
dividing the non-mold into adjacent working areas according to a second preset condition;
and filtering out the non-mold parts and the working area which do not accord with the second preset condition.
Further, the step of analyzing the working area meeting the first preset condition according to the original tracking position in the first video image and the preset tracking condition to generate an instruction for adjusting the original tracking position includes:
judging whether the working area meeting the first preset condition is located at the original tracking position or not according to the preset tracking condition;
if yes, the original tracking position is updated.
Further, the step of analyzing the working area meeting the first preset condition according to the original tracking position in the first video image and the preset tracking condition to generate an instruction for adjusting the original tracking position includes:
if not, judging whether the distance between the center of the mold part in the working area meeting the first preset condition and the original tracking position is smaller than a preset threshold value or not;
if yes, updating the central position;
if not, filtering the working area.
The invention has the beneficial effects that: and extracting the starting and ending time points of the operation steps of the workers, so as to obtain the technical indexes such as preheating time, preparation time, reaction time, sedation time, tumor pushing time and the like, whether the technical indexes are in place or not, and the like. The computer vision method replaces the manual operation. The labor cost is reduced, the efficiency is improved, all welding heads are scored, and the maintenance cost and the accident rate of the railway are reduced by performing key maintenance on the welding heads with poor quality. When a plurality of jobs are simultaneously operated in the screen, it is convenient to determine which work area of the current frame the detected part belongs to and which work area of the current frame and the previous frame are the same after the object detection. After the model is established through the targets, the detection results are subjected to rule optimization, targets of various types distributed at various positions in the picture are reasonably grouped, and when a plurality of welding jobs exist in the picture, the welding jobs are associated with the jobs in the same working area in the historical frame, namely, the results are subjected to correlation from two dimensions of space and time. Since the camera and the welding point location of the use scene hardly move, but the form of the welding area varies greatly, the problem can be solved better by tracking the key point in the present invention. The problem of video delivery railway administration back manpower audit inefficiency and can't accomplish fast and cause the backlog is solved. The problems are found in time with less labor and time cost, so that the accident rate is reduced, and the safe production quality is improved; the problems existing in the welding process are found by checking the video, such as whether a video exists or not, whether the video exists or not, preheating time, reaction time, sedation time, tumor pushing time and other technical indexes are in place or not; the shooting conditions and shooting specifications of track welding are standardized and systematized to form an industrial standard.
In addition, the present invention also provides a device for dividing and tracking a work area, comprising: the device comprises an acquisition device, a processing device and a control device, wherein the acquisition device is used for acquiring a first video image for evaluating the operation quality of a welding head, a second video image for evaluating the operation quality of the welding head in real time, a first preset condition and a preset tracking condition, and an original tracking position is preset in the first video image;
a processing device for performing an initialization of the original tracking position;
the processing equipment is also used for carrying out video frame extraction processing on the second video image to generate a third video image;
the processing equipment is also used for carrying out frame detection on the third video image to generate a fourth video image;
the processing device is further used for carrying out working area division processing on the fourth video image to generate a fifth video image with a plurality of working areas, wherein the plurality of working areas are respectively provided with a mould component;
the processing device is further used for screening the plurality of working areas in the fifth video image according to a first preset condition to generate a working area meeting the first preset condition;
and the processing equipment is also used for analyzing the working area which accords with the first preset condition according to the original tracking position in the first video image and the preset tracking condition, and generating an instruction for adjusting the original tracking position.
Further, the first preset condition is that: reserving a working area with the most three parts in a plurality of working areas;
the acquisition equipment is also used for acquiring the number of the parts contained in each working area in the fifth video image; wherein the parts comprise a mold part and a non-mold part;
the processing equipment is also used for sequencing the working areas according to the number of the parts contained in each working area to form a working area list with the number of the parts gradually reduced;
the processing device is further used for screening the three working areas with the top rank in the working area list and generating the working areas meeting the first preset condition.
Further, the obtaining device is further configured to obtain a second preset condition;
processing means for screening out mould parts and non-mould parts other than mould parts in the fourth video image;
a processing device further for creating a working area centered on the mold part;
the processing equipment is also used for dividing the non-mold into adjacent working areas according to a second preset condition;
and the processing equipment is also used for filtering out non-mold parts and working areas which do not accord with the second preset condition.
Further, the processing device is further configured to determine, according to a preset tracking condition, whether the working area meeting the first preset condition is located at the original tracking position;
and if so, updating the original tracking position.
Further, if the judgment result is negative, judging whether the distance between the center of the mold part in the working area meeting the first preset condition and the original tracking position is smaller than a preset threshold value;
if the judgment result is yes, updating the central position;
if the judgment result is negative, the working area is filtered.
The invention has the beneficial effects that: and extracting the starting and ending time points of the operation steps of the workers, so as to obtain the technical indexes such as preheating time, preparation time, reaction time, sedation time, tumor pushing time and the like, whether the technical indexes are in place or not, and the like. The computer vision method replaces the manual operation. The labor cost is reduced, the efficiency is improved, all welding heads are scored, and the maintenance cost and the accident rate of the railway are reduced by performing key maintenance on the welding heads with poor quality. When a plurality of jobs are simultaneously operated in the screen, it is convenient to determine which work area of the current frame the detected part belongs to and which work area of the current frame and the previous frame are the same after the object detection. After the model is established through the targets, the detection results are subjected to rule optimization, targets of various types distributed at various positions in the picture are reasonably grouped, and when a plurality of welding jobs exist in the picture, the welding jobs are associated with the jobs in the same working area in the historical frame, namely, the results are subjected to correlation from two dimensions of space and time. Since the camera and the welding point location of the use scene hardly move, but the form of the welding area varies greatly, the problem can be solved better by tracking the key point in the present invention. The problem of video delivery railway administration back manpower audit inefficiency and can't accomplish fast and cause the backlog is solved. The problems are found in time with less labor and time cost, so that the accident rate is reduced, and the safe production quality is improved; the problems existing in the welding process are found by checking the video, such as whether a video exists or not, whether the video exists or not, preheating time, reaction time, sedation time, tumor pushing time and other technical indexes are in place or not; the shooting conditions and shooting specifications of track welding are standardized and systematized to form an industrial standard.
Advantageous positions for additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
Fig. 1 is a schematic flowchart of a method for partitioning and tracking a workspace according to an embodiment of the present invention.
Fig. 2 is a second schematic flowchart of a method for partitioning and tracking a work area according to an embodiment of the present invention.
Fig. 3 is a third schematic flowchart of a method for partitioning and tracking a work area according to an embodiment of the present invention.
Fig. 4 is a fourth schematic flowchart of a workspace partitioning and tracking method according to an embodiment of the present invention.
Fig. 5 is a fifth schematic flowchart of a workspace partitioning and tracking method according to an embodiment of the present invention.
Fig. 6 is a schematic diagram illustrating a principle of a workspace partitioning and tracking method according to an embodiment of the present invention.
Fig. 7 is a schematic structural block diagram of a work area dividing and tracking device provided by an embodiment of the present invention.
Detailed Description
The principles and features of this invention are described below in conjunction with the following drawings, which are set forth by way of illustration only and are not intended to limit the scope of the invention.
As shown in fig. 1 to 6, an embodiment of the present invention provides a method for partitioning and tracking a workspace, including:
s1, acquiring a first video image for evaluating the operation quality of the welding head, a second video image for evaluating the operation quality of the welding head in real time, a first preset condition and a preset tracking condition, wherein an original tracking position is preset in the first video image;
s2, performing initialization on the original tracking position;
s3, performing video frame extraction processing on the second video image to generate a third video image;
s4, carrying out frame detection on the third video image to generate a fourth video image;
s5, carrying out working area division processing on the fourth video image to generate a fifth video image with a plurality of working areas, wherein the plurality of working areas are respectively provided with a mould component;
s6, screening a plurality of working areas in the fifth video image according to a first preset condition to generate a working area meeting the first preset condition;
and S7, analyzing the working area which meets the first preset condition according to the original tracking position in the first video image and the preset tracking condition, and generating an instruction for adjusting the original tracking position.
And adjusting the tracking position according to the instruction for adjusting the original tracking position.
The embodiment of the invention sets an original tracking position in an initial image of a video, then compares and analyzes the image of a subsequent video and the original tracking position of the initial image in real-time video monitoring, and then adjusts the tracking position according to a preset condition.
As shown in fig. 1 to 6, specifically, the original tracking point is initialized; video frame extraction; detecting model reasoning; a work area division module (device); sorting according to the number of the parts in the working areas, and selecting 3 working areas with the most parts; a workspace tracking module (device); classification and other follow-up analyses; and returning to the video frame extracting step.
Wherein the working area division module is used for: detecting and tracking the current frame; screening out the mould parts and creating an operation area; the other components are divided into operation areas with the shortest distance according to the Euclidean distance; filtering out parts with unreasonable relative positions based on the relative positions; the part for filtering the current frame only has the operation area of the mold.
The workspace tracking module is to: grouping the detection results; whether it is located at an initial position; if yes, updating the position of the central point; the center point is the original tracking position. If not, judging whether the distance between the center of the working area die and the central point is smaller than a threshold value; if yes, updating the position of the central point; if not, PASS is filtered. And subsequently, labeling the operation area according to the central point position label.
The problem of video delivery railway administration back manpower audit inefficiency and can't accomplish fast and cause the backlog is solved. The problems are expected to be found in time with less labor and time cost, so that the accident rate is reduced, and the safety production quality is improved; the problems existing in the welding process are expected to be found by checking videos, such as whether video recording exists or not, whether technical indexes such as preheating time, reaction time, calming time and tumor pushing time exist or not, and the like; the shooting conditions and shooting specifications of track welding are standardized and systematized to form an industrial standard.
The invention belongs to the field of video content analysis of computer vision, but has different purposes, and is mainly characterized in that the starting and ending time points of the operation steps of workers are extracted, so that the technical indexes of preheating time, preparation time, reaction time, sedation time, tumor pushing time and the like are obtained, and the like. No patents related to quality analysis of welding-based video are currently being looked up. The prior art scheme is that a special worker in a work section arranges people to manually check videos, and the method is low in efficiency, high in cost, unreliable in accuracy and generally overstocked.
As shown in fig. 1-6, annotating a data set;
selecting and training a deep learning target detection algorithm (yolov4, precision and speed optimal balance);
counting and tracking of the detection target;
dividing and tracking a working area;
judging the working area state based on the frame content;
judging the step nodes based on the whole video context;
the embodiment of the invention introduces the 4 th step, namely after the objects pass through the model, and the detection result is subjected to rule optimization, how to reasonably group various types of objects distributed at various positions in the picture, and when a plurality of welding operations exist in the picture, how to establish connection with the operations in the same working area in the historical frame, namely, the results are subjected to correlation from two dimensions of space and time.
As shown in fig. 1 to 6, the specific embodiment is sequentially performed according to the following steps:
division of a working area: detecting and tracking the current frame; screening out the mould parts and creating an operation area; the other components are divided into operation areas with the shortest distance according to the Euclidean distance; filtering out parts with unreasonable relative positions based on the relative positions; the part for filtering the current frame only has the operation area of the mold.
Screening all the moulds of the current frame, temporarily considering that each mould represents an operation area, because the moulds almost exist in the whole operation process and are fixed on the steel rail for reference, the false detection or the missing detection is almost avoided;
selecting the mold with the closest Euclidean distance for the other parts except the mold, and dividing the mold into the operation area;
filtering the parts in the operation area according to the positions of the relative die-checking;
the number of parts in the work area is filtered, and if only the mold is present, the work area may be mischecked or have no valid welding work.
The rule of the specific relative position is as shown in fig. 6: for the detected mold (iv), the width is W, the height is H, and the positions of other parts need to be defined as the relative positions as shown in the figure, as follows: the central limit range of the crucible cover, the central limit range of the crucible barrel, the left central limit range of the ash tray and the detected right central limit range of the ash tray of the mold.
The most accurate detection target can be ensured to be matched for each part, a working area M mark is added, and redundant detection, false detection targets and invalid targets are filtered.
Workspace tracking
Three original central points are initialized, and the coordinates are respectively: (0.25 w, 0), 0.5 w, 0), 0.75 w, 0), typically a maximum of three welding jobs are taken in one frame and three points are labeled L \ M \ R, as the three points on the edge on the graph;
if the center point is located at the original position, the updated distance is not limited. As shown in the figure, since the mold is installed, the position of the M center point is changed after the first detection;
and for the detection result of one frame, dividing the working area. Then, for each center point, the closest work area is selected. And if the center point is in the range of the rectangular boundary of the die in the operation area or the initial position of the center point, updating the position of the center point, wherein the new coordinate is the center point of the die in the working area, and marking the mark of the operation area as the center point.
The three central points are dynamically updated along with the operation area, and the operation area is endowed with a unique and unchangeable L/M/R mark.
The problem of video delivery railway administration back manpower audit inefficiency and can't accomplish fast and cause the backlog is solved. The problems are expected to be found in time with less labor and time cost, so that the accident rate is reduced, and the safety production quality is improved; the problems existing in the welding process are expected to be found by checking videos, such as whether video recording exists or not, whether technical indexes such as preheating time, reaction time, calming time and tumor pushing time exist or not, and the like; the shooting conditions and shooting specifications of track welding are standardized and systematized to form an industrial standard.
Similar effects can be achieved by using a traditional multi-target tracking method, but the method is relatively time-consuming and has high computational complexity. Since the camera and the welding point location using the scene hardly move, but the morphology of the welding area varies greatly, the tracking of the key point can solve the problem better. A working area division method based on the detection result (determining the attribution relation of the spatial dimension working area); and a working area tracking method (determining the corresponding relation of the time dimension working area).
The technical scheme of the embodiment of the invention aims at the target with real-time change of the form and carries out tracking processing on the target.
Further, the first preset condition is that: reserving a working area with the most three parts in a plurality of working areas;
the step of screening the plurality of working areas in the fifth video image according to the first preset condition to generate the working areas meeting the first preset condition includes:
acquiring the number of parts contained in each working area in the fifth video image; wherein the parts comprise a mold part and a non-mold part;
sorting the working areas according to the number of the parts contained in each working area to form a working area list with the number of the parts gradually reduced;
and screening three working areas with the top rank in the working area list to generate the working areas meeting the first preset condition.
Further, the step of performing work area division processing on the fourth video image to generate a fifth video image having a plurality of work areas includes:
acquiring a second preset condition;
screening the mold parts and non-mold parts other than the mold parts in the fourth video image;
creating a working area centered on the mold part;
dividing the non-mold into adjacent working areas according to a second preset condition;
and filtering out the non-mold parts and the working area which do not accord with the second preset condition.
Further, the step of analyzing the working area meeting the first preset condition according to the original tracking position in the first video image and the preset tracking condition to generate an instruction for adjusting the original tracking position includes:
judging whether the working area meeting the first preset condition is located at the original tracking position or not according to the preset tracking condition;
if yes, the original tracking position is updated.
Further, the step of analyzing the working area meeting the first preset condition according to the original tracking position in the first video image and the preset tracking condition to generate an instruction for adjusting the original tracking position includes:
if not, judging whether the distance between the center of the mold part in the working area meeting the first preset condition and the original tracking position is smaller than a preset threshold value or not;
if yes, updating the central position;
if not, filtering the working area.
The invention has the beneficial effects that: and extracting the starting and ending time points of the operation steps of the workers, so as to obtain the technical indexes such as preheating time, preparation time, reaction time, sedation time, tumor pushing time and the like, whether the technical indexes are in place or not, and the like. The computer vision method replaces the manual operation. The labor cost is reduced, the efficiency is improved, all welding heads are scored, and the maintenance cost and the accident rate of the railway are reduced by performing key maintenance on the welding heads with poor quality. When a plurality of jobs are simultaneously operated in the screen, it is convenient to determine which work area of the current frame the detected part belongs to and which work area of the current frame and the previous frame are the same after the object detection. After the model is established through the targets, the detection results are subjected to rule optimization, targets of various types distributed at various positions in the picture are reasonably grouped, and when a plurality of welding jobs exist in the picture, the welding jobs are associated with the jobs in the same working area in the historical frame, namely, the results are subjected to correlation from two dimensions of space and time. Since the camera and the welding point location of the use scene hardly move, but the form of the welding area varies greatly, the problem can be solved better by tracking the key point in the present invention. The problem of video delivery railway administration back manpower audit inefficiency and can't accomplish fast and cause the backlog is solved. The problems are found in time with less labor and time cost, so that the accident rate is reduced, and the safe production quality is improved; the problems existing in the welding process are found by checking the video, such as whether a video exists or not, whether the video exists or not, preheating time, reaction time, sedation time, tumor pushing time and other technical indexes are in place or not; the shooting conditions and shooting specifications of track welding are standardized and systematized to form an industrial standard.
As shown in fig. 7, the present invention further provides a work area dividing and tracking apparatus, which includes: the device comprises an acquisition device, a processing device and a control device, wherein the acquisition device is used for acquiring a first video image for evaluating the operation quality of a welding head, a second video image for evaluating the operation quality of the welding head in real time, a first preset condition and a preset tracking condition, and an original tracking position is preset in the first video image;
a processing device for performing an initialization of the original tracking position;
the processing equipment is also used for carrying out video frame extraction processing on the second video image to generate a third video image;
the processing equipment is also used for carrying out frame detection on the third video image to generate a fourth video image;
the processing device is further used for carrying out working area division processing on the fourth video image to generate a fifth video image with a plurality of working areas, wherein the plurality of working areas are respectively provided with a mould component;
the processing device is further used for screening the plurality of working areas in the fifth video image according to a first preset condition to generate a working area meeting the first preset condition;
and the processing equipment is also used for analyzing the working area which accords with the first preset condition according to the original tracking position in the first video image and the preset tracking condition, and generating an instruction for adjusting the original tracking position.
Further, the first preset condition is that: reserving a working area with the most three parts in a plurality of working areas;
the acquisition equipment is also used for acquiring the number of the parts contained in each working area in the fifth video image; wherein the parts comprise a mold part and a non-mold part;
the processing equipment is also used for sequencing the working areas according to the number of the parts contained in each working area to form a working area list with the number of the parts gradually reduced;
the processing device is further used for screening the three working areas with the top rank in the working area list and generating the working areas meeting the first preset condition.
Further, the obtaining device is further configured to obtain a second preset condition;
processing means for screening out mould parts and non-mould parts other than mould parts in the fourth video image;
a processing device further for creating a working area centered on the mold part;
the processing equipment is also used for dividing the non-mold into adjacent working areas according to a second preset condition;
and the processing equipment is also used for filtering out non-mold parts and working areas which do not accord with the second preset condition.
Further, the processing device is further configured to determine, according to a preset tracking condition, whether the working area meeting the first preset condition is located at the original tracking position;
and if so, updating the original tracking position.
Further, if the judgment result is negative, judging whether the distance between the center of the mold part in the working area meeting the first preset condition and the original tracking position is smaller than a preset threshold value;
if the judgment result is yes, updating the central position;
if the judgment result is negative, the working area is filtered.
The invention has the beneficial effects that: and extracting the starting and ending time points of the operation steps of the workers, so as to obtain the technical indexes such as preheating time, preparation time, reaction time, sedation time, tumor pushing time and the like, whether the technical indexes are in place or not, and the like. The computer vision method replaces the manual operation. The labor cost is reduced, the efficiency is improved, all welding heads are scored, and the maintenance cost and the accident rate of the railway are reduced by performing key maintenance on the welding heads with poor quality. When a plurality of jobs are simultaneously operated in the screen, it is convenient to determine which work area of the current frame the detected part belongs to and which work area of the current frame and the previous frame are the same after the object detection. After the model is established through the targets, the detection results are subjected to rule optimization, targets of various types distributed at various positions in the picture are reasonably grouped, and when a plurality of welding jobs exist in the picture, the welding jobs are associated with the jobs in the same working area in the historical frame, namely, the results are subjected to correlation from two dimensions of space and time. Since the camera and the welding point location of the use scene hardly move, but the form of the welding area varies greatly, the problem can be solved better by tracking the key point in the present invention. The problem of video delivery railway administration back manpower audit inefficiency and can't accomplish fast and cause the backlog is solved. The problems are found in time with less labor and time cost, so that the accident rate is reduced, and the safe production quality is improved; the problems existing in the welding process are found by checking the video, such as whether a video exists or not, whether the video exists or not, preheating time, reaction time, sedation time, tumor pushing time and other technical indexes are in place or not; the shooting conditions and shooting specifications of track welding are standardized and systematized to form an industrial standard.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for partitioning and tracking a workspace, comprising:
acquiring a first video image for evaluating the operation quality of a welding head, a second video image for evaluating the operation quality of the welding head in real time, a first preset condition and a preset tracking condition, wherein an original tracking position is preset in the first video image;
the original tracking position is subjected to the initialization;
performing video frame extraction processing on the second video image to generate a third video image;
performing frame detection on the third video image to generate a fourth video image;
performing working area division processing on the fourth video image to generate a fifth video image with a plurality of working areas, wherein the plurality of working areas are respectively provided with a mould component;
screening a plurality of working areas in the fifth video image according to a first preset condition to generate a working area meeting the first preset condition;
and analyzing the working area which accords with the first preset condition according to the original tracking position in the first video image and the preset tracking condition to generate an instruction for adjusting the original tracking position.
2. The method for partitioning and tracking a workspace according to claim 1, wherein the first preset condition is: reserving a working area with the most three parts in a plurality of working areas;
the step of screening the plurality of working areas in the fifth video image according to the first preset condition to generate the working areas meeting the first preset condition includes:
acquiring the number of parts contained in each working area in the fifth video image; wherein the parts comprise a mold part and a non-mold part;
sorting the working areas according to the number of the parts contained in each working area to form a working area list with the number of the parts gradually reduced;
and screening three working areas with the top rank in the working area list to generate the working areas meeting the first preset condition.
3. The method for dividing and tracking a workspace according to claim 1, wherein the step of performing workspace division processing on the fourth video image to generate a fifth video image having a plurality of workspaces comprises:
acquiring a second preset condition;
screening the mold parts and non-mold parts other than the mold parts in the fourth video image;
creating a working area centered on the mold part;
dividing the non-mold into adjacent working areas according to a second preset condition;
and filtering out the non-mold parts and the working area which do not accord with the second preset condition.
4. The method for dividing and tracking a workspace according to claim 1, wherein the step of analyzing the workspace meeting the first preset condition according to the original tracking position in the first video image and the preset tracking condition to generate the instruction for adjusting the original tracking position comprises:
judging whether the working area meeting the first preset condition is located at the original tracking position or not according to the preset tracking condition;
if yes, the original tracking position is updated.
5. The method for dividing and tracking a workspace according to claim 4, wherein the step of analyzing the workspace meeting the first preset condition according to the original tracking position in the first video image and the preset tracking condition to generate the instruction for adjusting the original tracking position comprises:
if not, judging whether the distance between the center of the mold part in the working area meeting the first preset condition and the original tracking position is smaller than a preset threshold value or not;
if yes, updating the central position;
if not, filtering the working area.
6. A workspace partitioning and tracking apparatus, comprising:
the device comprises an acquisition device, a processing device and a control device, wherein the acquisition device is used for acquiring a first video image for evaluating the operation quality of a welding head, a second video image for evaluating the operation quality of the welding head in real time, a first preset condition and a preset tracking condition, and an original tracking position is preset in the first video image;
a processing device for performing an initialization of the original tracking position;
the processing equipment is also used for carrying out video frame extraction processing on the second video image to generate a third video image;
the processing equipment is also used for carrying out frame detection on the third video image to generate a fourth video image;
the processing device is further used for carrying out working area division processing on the fourth video image to generate a fifth video image with a plurality of working areas, wherein the plurality of working areas are respectively provided with a mould component;
the processing device is further used for screening the plurality of working areas in the fifth video image according to a first preset condition to generate a working area meeting the first preset condition;
and the processing equipment is also used for analyzing the working area which accords with the first preset condition according to the original tracking position in the first video image and the preset tracking condition, and generating an instruction for adjusting the original tracking position.
7. The work area division and tracking device according to claim 6, wherein the first preset condition is: reserving a working area with the most three parts in a plurality of working areas;
the acquisition equipment is also used for acquiring the number of the parts contained in each working area in the fifth video image; wherein the parts comprise a mold part and a non-mold part;
the processing equipment is also used for sequencing the working areas according to the number of the parts contained in each working area to form a working area list with the number of the parts gradually reduced;
the processing device is further used for screening the three working areas with the top rank in the working area list and generating the working areas meeting the first preset condition.
8. A workspace splitting and tracking mechanism as in claim 6,
the acquisition equipment is also used for acquiring a second preset condition;
processing means for screening out mould parts and non-mould parts other than mould parts in the fourth video image;
a processing device further for creating a working area centered on the mold part;
the processing equipment is also used for dividing the non-mold into adjacent working areas according to a second preset condition;
and the processing equipment is also used for filtering out non-mold parts and working areas which do not accord with the second preset condition.
9. A workspace splitting and tracking mechanism as in claim 6,
the processing equipment is also used for judging whether the working area meeting the first preset condition is positioned at the original tracking position or not according to the preset tracking condition;
and if so, updating the original tracking position.
10. The work area partitioning and tracking apparatus as claimed in claim 9,
if not, judging whether the distance between the center of the mold part in the working area meeting the first preset condition and the original tracking position is smaller than a preset threshold value or not;
if the judgment result is yes, updating the central position;
if the judgment result is negative, the working area is filtered.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110544127.XA CN113177932B (en) | 2021-05-19 | 2021-05-19 | Method and device for dividing and tracking working areas |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110544127.XA CN113177932B (en) | 2021-05-19 | 2021-05-19 | Method and device for dividing and tracking working areas |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113177932A true CN113177932A (en) | 2021-07-27 |
| CN113177932B CN113177932B (en) | 2023-08-04 |
Family
ID=76929472
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110544127.XA Active CN113177932B (en) | 2021-05-19 | 2021-05-19 | Method and device for dividing and tracking working areas |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113177932B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0929434A (en) * | 1995-07-14 | 1997-02-04 | Suzuki Motor Corp | Welding position inspecting apparatus and welding system |
| JP2006038784A (en) * | 2004-07-30 | 2006-02-09 | Suncall Corp | Method and device for measuring weld position of magnetic head suspension |
| US20180130376A1 (en) * | 2016-11-07 | 2018-05-10 | Lincoln Global, Inc. | Welding trainer utilizing a head up display to display simulated and real-world objects |
| CN108057938A (en) * | 2016-11-07 | 2018-05-22 | 林肯环球股份有限公司 | The welding system that vision and audio prompt are provided welder's helmet with display |
| CN110268441A (en) * | 2017-01-09 | 2019-09-20 | 凯尔索富特环球控股有限公司 | The method for obtaining the 3D model data of multiple components of object |
| JP2019214100A (en) * | 2018-06-13 | 2019-12-19 | オムロン株式会社 | Robot control device, robot control method, and robot control program |
| CN111761580A (en) * | 2020-07-01 | 2020-10-13 | 青岛九维华盾科技研究院有限公司 | Multi-robot pattern spraying working area flexible segmentation method |
-
2021
- 2021-05-19 CN CN202110544127.XA patent/CN113177932B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0929434A (en) * | 1995-07-14 | 1997-02-04 | Suzuki Motor Corp | Welding position inspecting apparatus and welding system |
| JP2006038784A (en) * | 2004-07-30 | 2006-02-09 | Suncall Corp | Method and device for measuring weld position of magnetic head suspension |
| US20180130376A1 (en) * | 2016-11-07 | 2018-05-10 | Lincoln Global, Inc. | Welding trainer utilizing a head up display to display simulated and real-world objects |
| CN108057938A (en) * | 2016-11-07 | 2018-05-22 | 林肯环球股份有限公司 | The welding system that vision and audio prompt are provided welder's helmet with display |
| CN110268441A (en) * | 2017-01-09 | 2019-09-20 | 凯尔索富特环球控股有限公司 | The method for obtaining the 3D model data of multiple components of object |
| JP2019214100A (en) * | 2018-06-13 | 2019-12-19 | オムロン株式会社 | Robot control device, robot control method, and robot control program |
| CN111761580A (en) * | 2020-07-01 | 2020-10-13 | 青岛九维华盾科技研究院有限公司 | Multi-robot pattern spraying working area flexible segmentation method |
Non-Patent Citations (1)
| Title |
|---|
| 张培耘;华希俊;倪健;符永宏;纪敬虎;: "微观复合造型模具拉延成形数值模拟和优化研究", 润滑与密封, no. 11, pages 34 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113177932B (en) | 2023-08-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111951237B (en) | Visual appearance detection method | |
| CN109389161B (en) | Garbage identification evolutionary learning method, device, system and medium based on deep learning | |
| CN111709935B (en) | Real-time coal gangue positioning and identifying method for ground moving belt | |
| CN104637058B (en) | A kind of volume of the flow of passengers identify statistical methods based on image information | |
| CN110738127A (en) | Helmet identification method based on unsupervised deep learning neural network algorithm | |
| CN113723325B (en) | Frock defect detecting system of prefabricated component of assembled | |
| CN117557570B (en) | Rail vehicle abnormality detection method and system | |
| CN106250846A (en) | A kind of public security image method for detecting based on video monitoring | |
| CN117392133B (en) | Die casting burr detection system and method based on machine vision | |
| CN114581760A (en) | Equipment fault detection method and system for machine room inspection | |
| CN113052125B (en) | Construction site violation image recognition and alarm method | |
| CN113177932A (en) | Method and device for dividing and tracking working area | |
| CN111368833B (en) | Full-automatic steel loading method for detecting slab number based on machine vision | |
| CN112541367A (en) | Multiple two-dimensional code identification method based on deep learning and image processing | |
| CN111178244A (en) | Method for identifying abnormal production scene | |
| CN113723841B (en) | On-line detection method for tool missing in assembled prefabricated part | |
| CN113420839B (en) | Semi-automatic labeling method and segmentation positioning system for stacking planar target objects | |
| CN205034418U (en) | Automatic branch steel device calmly of rod based on machine vision | |
| CN114821364A (en) | Deepsort garden garbage statistical method based on unmanned aerial vehicle motion state | |
| CN115063448A (en) | Multi-target tracking method for oilfield operation scene | |
| CN114049543A (en) | Automatic identification method for scrap steel unloading change area based on deep learning | |
| CN113177931A (en) | Method and device for detecting and tracking key component | |
| CN112149499A (en) | Wood block direction recognition device based on machine vision technology | |
| CN117911415B (en) | Automatic equipment supervision system and method based on machine vision | |
| CN117541739B (en) | Warehouse map visual construction method and system based on OpenCV |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220728 Address after: Room 07, 6th floor, building 6, District 4, No. 186, South Fourth Ring West Road, Fengtai District, Beijing 100160 Applicant after: Beijing Xiaoming zhitie Technology Co.,Ltd. Address before: 100089 a1002, 10th floor, building 1, yard 1, Zhongguancun East Road, Haidian District, Beijing Applicant before: MININGLAMP SOFTWARE SYSTEMS Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |