CN112927247A - Graph cutting method based on target detection, graph cutting device and storage medium - Google Patents
Graph cutting method based on target detection, graph cutting device and storage medium Download PDFInfo
- Publication number
- CN112927247A CN112927247A CN202110248957.8A CN202110248957A CN112927247A CN 112927247 A CN112927247 A CN 112927247A CN 202110248957 A CN202110248957 A CN 202110248957A CN 112927247 A CN112927247 A CN 112927247A
- Authority
- CN
- China
- Prior art keywords
- target
- cutting
- map
- image
- strategy
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/12—Edge-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Abstract
The invention provides a graph cutting method, a graph cutting device and a storage medium based on target detection, wherein the method comprises the following steps: acquiring position information and size of a target frame in a detected image to judge the distribution rule of the target frame; judging the distribution condition of the target in the detection image; selecting a map cutting strategy according to the distribution rule of the target frame and the distribution condition of the target in the detected image, and cutting the map according to the map cutting strategy; inputting the cut picture into a deep learning model for training to judge the cut picture effect; selecting Negative chips containing difficult Negative samples according to the cutting effect, carrying out region extraction on the Negative samples, and cutting according to the extracted regions. According to the method, an appropriate image cutting strategy can be selected for automatic image cutting according to the target, the position, the size and the distribution condition, and the image cutting effect is further evaluated by combining with the subsequent deep learning model expression, so that the image cutting effect is ensured, and a foundation is laid for the subsequent actual operation.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a target detection-based map cutting method, a target detection-based map cutting device and a non-transitory computer-readable storage medium.
Background
In the learning and detection of the depth algorithm based on the target detection, image data generally needs to be subjected to image cutting processing to meet various algorithm requirements or machine memory requirements, such as 32 times of yolo, for example, the image data is cut into the sizes of 512 × 512, 256 × 256 and the like. The image is reasonably cut, accurate learning of all targets is guaranteed, extra offset information is not added, and model learning is more effective, so that the method has important significance for expression of the algorithm.
Therefore, how to efficiently cut the graph, retain and enhance the effective information to the maximum extent, and reduce the unnecessary redundancy requirement becomes a technical problem to be solved by those skilled in the art.
Disclosure of Invention
The invention provides a graph cutting method based on target detection for solving the technical problems, and the method can select a proper graph cutting strategy to automatically cut graphs according to the target, the position, the size and the distribution condition, and further evaluate the graph cutting effect by combining with the subsequent deep learning model expression, thereby ensuring the graph cutting effect and laying a foundation for the subsequent actual operation.
The invention also provides a graph cutting device based on target detection.
The invention also proposes a non-transitory computer-readable storage medium.
The technical scheme adopted by the invention is as follows:
the embodiment of the first aspect of the invention provides a graph cutting method based on target detection, which comprises the following steps: acquiring position information and size of a target frame in a detection image, and judging a distribution rule of the target frame according to the position information and the size of the target frame; counting the non-black non-white area and the pixel skipping condition in the detection image, and judging the distribution condition of the target in the detection image according to the non-black non-white area and the pixel skipping condition; selecting a map cutting strategy according to the distribution rule of the target frame and the distribution condition of the target in the detection image, and cutting the map according to the map cutting strategy, wherein the map cutting strategy comprises the following steps: the method comprises the following steps of (1) a sliding window method map cutting strategy, a special position map cutting strategy, a clustering map cutting strategy and an edge-based map cutting strategy; inputting the picture after the picture cutting into a deep learning model for training so as to judge the picture cutting effect; selecting Negative chips containing difficult Negative samples according to the cutting effect, carrying out region extraction on the Negative samples, and cutting according to the extracted regions.
According to an embodiment of the present invention, further comprising: acquiring the distribution rule change of a target frame of the picture after the picture is cut and the size change of the target frame; inputting the image after the image cutting into a single-stage target detection model or a double-stage target detection model to obtain the algorithm expression of the image after the image cutting, wherein the algorithm expression comprises the following steps: the over-killing rate of the target, the missing rate of the target, the accuracy rate of the target, a confusion matrix of the target, a class AP (Average Precision) value and an overall mAP (mean Average Precision) value; and correcting the graph cutting strategy according to the distribution rule change of the target frame, the size change of the target frame and the algorithm performance.
According to one embodiment of the invention, the single-stage object detection model comprises: YOLOv5 model, SSD (Single Shot multi box Detector) model, the two-stage object detection model comprising: fast R-CNN (Region probabilistic Neural Networks based on Convolutional Neural Networks), Cascade R-CNN, HTC (Hybrid Task Cascade).
According to one embodiment of the invention, based on the map cutting effect, selecting Negative chips maps containing hard Negative samples based on SNIPER algorithm.
According to an embodiment of the present invention, selecting a graph cutting strategy according to a distribution rule of the target frame and a distribution condition of a target in the detection image includes: if the target frames are distributed evenly and the detection image contains all information of the target, a window method image cutting strategy is adopted for cutting the image; if the target frames are distributed evenly and the detection image contains partial information of the target, adopting a special position map cutting strategy; if the target frame is unevenly distributed and the detection image contains all information of the target, adopting an edge-based map cutting strategy; and if the target frames are not distributed uniformly and the detection image contains partial information of the target, adopting a clustering and graph cutting strategy.
The embodiment of the second aspect of the invention provides a graph cutting device based on target detection, which comprises: the acquisition module is used for acquiring the position information and the size of a target frame in a detection image and judging the distribution rule of the target frame according to the position information and the size of the target frame; the statistical module is used for counting the non-black non-white area and the pixel jumping condition in the detection image and judging the distribution condition of the target in the detection image according to the non-black non-white area and the pixel jumping condition; the first map cutting module is used for selecting a map cutting strategy according to the distribution rule of the target frame and the distribution condition of the target in the detection image, and cutting the map according to the map cutting strategy, wherein the map cutting strategy comprises the following steps: the method comprises the following steps of (1) a sliding window method map cutting strategy, a special position map cutting strategy, a clustering map cutting strategy and an edge-based map cutting strategy; the learning module is used for inputting the cut pictures into a deep learning model for training so as to judge the cut effect; and the second map cutting module is used for picking out Negative chips containing difficult Negative samples according to the map cutting effect, carrying out region extraction on the Negative samples and carrying out map cutting according to the extracted regions.
According to an embodiment of the present invention, the above graph cutting apparatus further includes a correction module, where the correction module is configured to: acquiring the distribution rule change of a target frame of the picture after the picture is cut and the size change of the target frame; inputting the image after the image cutting into a single-stage target detection model or a double-stage target detection model to obtain the algorithm expression of the image after the image cutting, wherein the algorithm expression comprises the following steps: the over-killing rate of the target, the omission factor of the target, the accuracy rate of the target, a confusion matrix of the target, a class AP value and an overall mAP value; and correcting the graph cutting strategy according to the distribution rule change of the target frame, the size change of the target frame and the algorithm performance.
According to an embodiment of the present invention, the second graph cutting module is specifically configured to: and selecting Negative chips containing hard Negative samples based on a SNIPER algorithm according to the cutting effect.
According to an embodiment of the present invention, the first cut-map module is specifically configured to: if the target frames are distributed evenly and the detection image contains all information of the target, a window method image cutting strategy is adopted for cutting the image; if the target frames are distributed evenly and the detection image contains partial information of the target, adopting a special position map cutting strategy; if the target frame is unevenly distributed and the detection image contains all information of the target, adopting an edge-based map cutting strategy; and if the target frames are not distributed uniformly and the detection image contains partial information of the target, adopting a clustering and graph cutting strategy.
An embodiment of the third aspect of the present invention provides a non-transitory computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the target detection-based graph cutting method according to the embodiment of the first aspect of the present invention.
The invention has the beneficial effects that:
according to the invention, through the target, the position, the size and the distribution condition, a proper image cutting strategy can be selected for automatic image cutting, and the image cutting effect is further evaluated by combining with the subsequent deep learning model expression, so that the image cutting effect is ensured, and a foundation is laid for the subsequent actual operation.
Drawings
FIG. 1 is a flow diagram of a graph cut method based on object detection according to one embodiment of the present invention;
FIG. 2 is a flow diagram of a graph cut method based on object detection according to another embodiment of the present invention;
fig. 3 is a block schematic diagram of a map cutting apparatus based on object detection according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a flowchart of a graph cutting method based on object detection according to an embodiment of the present invention, the method comprising the steps of:
and S1, acquiring the position information and the size of the target frame in the detected image, and judging the distribution rule of the target frame according to the position information and the size of the target frame.
Specifically, the target detection image has a target frame for labeling the target, for example, defects in the industrial quality inspection image are labeled in the form of the target frame. For example, if the target frames are distributed in each area of the whole graph and the size and the number of the targets distributed in each area are similar, it may be determined that the distribution of the target frames in the whole graph is balanced, or the target frames are distributed only in a partial area, and it may be determined that the distribution of the targets in the whole graph is unbalanced.
And S2, counting the non-black and non-white areas and the pixel skipping condition in the detection image, and judging the distribution condition of the target in the detection image according to the non-black and non-white areas and the pixel skipping condition.
Specifically, the edge of the workpiece can be determined according to the non-black and non-white area and the pixel skipping condition in the image, and the distribution condition of the target in the detected image can be determined according to the edge, wherein the distribution condition comprises whether the workpiece occupies the full image.
S3, selecting a map cutting strategy according to the distribution rule of the target frame and the distribution condition of the target in the detected image, and cutting the map according to the map cutting strategy, wherein the map cutting strategy comprises: the method comprises a sliding window method map cutting strategy, a special position map cutting strategy, a clustering map cutting strategy and an edge-based map cutting strategy.
Further, according to an embodiment of the present invention, selecting a graph cutting strategy according to a distribution rule of a target frame and a distribution condition of a target in a detected image, includes: if the target frames are distributed evenly and the detected image contains all information of the target, adopting a window method map cutting strategy to cut the map; if the target frames are distributed evenly and the detected image contains partial information of the target, adopting a special position map cutting strategy; if the target frames are not distributed uniformly and the detected image contains all information of the target, adopting an edge-based map cutting strategy; and if the target frames are unevenly distributed and the detected image contains partial information of the target, adopting a clustering and graph cutting strategy.
And S4, inputting the picture after the picture cutting into the deep learning model for training to judge the picture cutting effect.
The deep learning model can be a neural network model, and the deep learning module can be trained in advance according to actual requirements.
And S5, selecting Negative chips containing difficult Negative samples according to the cutting effect, carrying out region extraction on the Negative samples, and cutting according to the extracted regions.
In an embodiment of the present invention, a Negative chips containing hard Negative samples is selected based on a SNIPER (Scale Normalization for Image pyramides with Efficient training algorithm) algorithm according to the graph cutting effect. The SNIPER algorithm selects Negative chips containing hard Negative samples as the prior art, and detailed description is omitted in the invention.
Specifically, the distribution rule of the target frame is judged according to the position information and the size of the target frame, then, the non-black non-white area and the pixel jumping situation in the detected image are counted, the distribution situation of the target in the detected image is judged according to the non-black non-white area and the pixel jumping situation, then, a picture cutting strategy is selected according to the distribution rule of the target frame and the distribution situation of the target in the detected image, and the picture is cut according to the picture cutting strategy, wherein if the target frame is distributed evenly, and the detected image contains all information of the target, the picture is cut by adopting a window method picture cutting strategy; if the target frames are distributed evenly and the detected image contains partial information of the target, adopting a special position map cutting strategy; if the target frames are not distributed uniformly and the detected image contains all information of the target, adopting an edge-based map cutting strategy; and if the target frames are unevenly distributed and the detected image contains partial information of the target, adopting a clustering and graph cutting strategy.
And cutting the picture according to the selected picture cutting strategy, inputting the picture after picture cutting into a deep learning model for training, and judging the picture cutting effect. Then, selecting Negative chips containing difficult Negative samples according to the cutting effect, screening the Negative difficult samples, extracting the Negative difficult samples through fine cutting, and replacing original pictures, thereby further improving the model expression and optimization efficiency. Therefore, according to the method, an appropriate image cutting strategy can be selected for automatic image cutting according to the target, the position, the size and the distribution condition, and the image cutting effect is further evaluated by combining with the subsequent deep learning model expression, so that the image cutting effect is ensured, and a foundation is laid for the subsequent actual operation.
In the invention, for the sliding window method map cutting strategy, firstly, the picture is cut from left to right and from top to bottom according to the preset map cutting size and filling position. Here, the judgment of the edge is realized, and the consistency of the size is ensured; in addition, the pictures can be filled according to requirements, and the filling method comprises mean value filling and edge filling.
For the special position map cutting strategy, the strategy is suitable for the condition that the main target is unevenly distributed. The specific scenes comprise: the picture is distributed in the center or the bottom or in the transverse direction, and the rest part is completely black or white, so that no effective information is provided for model learning, and the cutting is not necessary. The special position cutting strategy realizes the targeted cutting of the target object at the special position.
For the Clustering and graph-cutting strategy, firstly, the position, the size and the length-width proportion of a target frame are counted, and the target frame, namely, the defects are clustered based on two methods, namely K-Means (Euclidean distance-based Clustering algorithm) and Spectral Clustering (Spectral Clustering) in machine learning according to the characteristic distribution of data.
For the edge-based cutting chart strategy, the strategy is constructed for enhancing cutting effectiveness aiming at pictures with irregular distribution or variable shooting angles. Firstly, identifying a corresponding edge according to pixel transformation and edge judgment, then implementing corresponding graph cutting according to the edge, and selecting whether a target is required to be ensured in the middle or not according to requirements.
According to an embodiment of the present invention, as shown in fig. 2, the above graph cutting method based on target detection further includes:
and S31, acquiring the distribution rule change of the target frame and the size change of the target frame of the picture after the picture is cut.
S32, inputting the picture after the picture is cut into a single-stage target detection model or a double-stage target detection model to obtain the algorithm expression of the picture after the picture is cut, wherein the algorithm expression comprises the following steps: the over-killing rate of the target, the missing detection rate of the target, the accuracy rate of the target, a confusion matrix of the target, a class AP value, an overall mAP value and the like.
The single-stage target detection model comprises: YOLOv5 model, SSD model, two-stage target detection model includes: fast R-CNN, Cascade R-CNN and HTC.
And S33, correcting the graph cutting strategy according to the distribution rule change of the target frame, the size change of the target frame and the algorithm expression.
Specifically, before the cut picture is input into the deep learning model for training, the cut picture may be input into a simple target detection model (for example, a single-stage model with a fast operation speed and a certain accuracy) in advance for pre-screening, so as to filter unnecessary samples and improve the subsequent deep learning effect.
Therefore, in the invention, after the graph cutting strategy is selected according to the distribution rule of the target frame and the distribution situation of the target in the detected image, and the graph cutting is carried out according to the graph cutting strategy, the distribution rule of the target frame of the picture after the graph cutting is obtained changes, the size of the target frame changes, and the graph cutting strategy is corrected according to the distribution rule change of the target frame and the size change of the target frame, for example, for the situations that the picture is too large and the target is too small, such as the picture with more than 5000 pixels and the target with the size of 50 pixels, if the background is not complex, the clustering graph cutting is considered to be used after the graph cutting of the window, the aggregation screenshot enhancement is carried out on the adjacent targets, and the model expression is improved. And inputting the cut pictures into a machine learning model to obtain the killing rate, the missing detection rate, the accuracy rate and the confusion matrix of the target, and correcting the cutting strategy according to the killing rate, the missing detection rate, the accuracy rate and the confusion matrix, for example, if the killing rate is high, the missing detection rate is high, the accuracy rate is low or the accuracy rate of the confusion matrix is low, correcting the cutting strategy. Meanwhile, the picture can be processed according to the algorithm representation of the picture after the picture is cut, such as background filling and the like.
Therefore, the picture cutting strategy aiming at different conditions is innovatively realized, unnecessary samples are filtered, the condition that no target frame exists after the picture is cut is judged and screened, and meanwhile, the picture condition can be filled and processed.
According to the image cutting method based on target detection provided by the embodiment of the invention, the position information and the size of a target frame in a detected image are obtained, the distribution rule of the target frame is judged according to the position information and the size of the target frame, the non-black non-white area and the pixel jumping condition in the detected image are counted, the distribution condition of a target in the detected image is judged according to the non-black non-white area and the pixel jumping condition, an image cutting strategy is selected according to the distribution rule of the target frame and the distribution condition of the target in the detected image, and image cutting is carried out according to the image cutting strategy, wherein the image cutting strategy comprises the following steps: the method comprises the steps of inputting a picture after picture cutting into a deep learning model for training to judge the picture cutting effect, selecting Negative chips containing difficult Negative samples according to the picture cutting effect, carrying out region extraction on the Negative samples, and cutting the picture according to the extracted region. Therefore, according to the method, an appropriate image cutting strategy can be selected for automatic image cutting according to the target, the position, the size and the distribution condition, and the image cutting effect is further evaluated by combining with the subsequent deep learning model expression, so that the image cutting effect is ensured, and a foundation is laid for the subsequent actual operation.
Corresponding to the graph cutting method based on the target detection, the invention also provides a graph cutting device based on the target detection. Since the device embodiment of the present invention corresponds to the method embodiment described above, details that are not disclosed in the device embodiment may refer to the method embodiment described above, and are not described again in the present invention.
Fig. 3 is a block schematic diagram of a map cutting apparatus based on object detection according to an embodiment of the present invention. As shown in fig. 3, the apparatus includes: the device comprises an acquisition module 1, a statistic module 2, a first map cutting module 3, a learning module 4 and a second map cutting module 5.
The acquisition module 1 is used for acquiring the position information and the size of a target frame in a detection image and judging the distribution rule of the target frame according to the position information and the size of the target frame; the statistical module 2 is used for counting the non-black non-white area and the pixel jumping condition in the detection image and judging the distribution condition of the target in the detection image according to the non-black non-white area and the pixel jumping condition; the first map cutting module 3 is configured to select a map cutting strategy according to a distribution rule of the target frame and a distribution condition of the target in the detected image, and perform map cutting according to the map cutting strategy, where the map cutting strategy includes: the method comprises the following steps of (1) a sliding window method map cutting strategy, a special position map cutting strategy, a clustering map cutting strategy and an edge-based map cutting strategy; the learning module 4 is used for inputting the cut pictures into a deep learning model for training so as to judge the cut picture effect; the second map cutting module 5 is used for selecting Negative chips containing difficult Negative samples according to the map cutting effect, extracting the areas of the Negative samples, and cutting the maps according to the extracted areas.
According to an embodiment of the present invention, the above graph cutting apparatus based on target detection further includes a modification module, where the modification module is configured to: acquiring the distribution rule change of a target frame of the picture after the picture is cut and the size change of the target frame; inputting the picture after the picture is cut into a single-stage target detection model or a double-stage target detection model to obtain the algorithm expression of the picture after the picture is cut, wherein the algorithm expression comprises the following steps: the over-killing rate of the target, the missing detection rate of the target, the accuracy rate of the target, a confusion matrix of the target, a class AP value, an overall mAP value and the like; and correcting the graph cutting strategy according to the distribution rule change of the target frame, the size change of the target frame and the algorithm expression.
The single-stage target detection model comprises: YOLOv5 model, SSD model, two-stage target detection model includes: fast R-CNN, Cascade R-CNN and HTC.
According to an embodiment of the present invention, the second graph cutting module 5 is specifically configured to: and selecting Negative chips containing hard Negative samples based on a SNIPER algorithm according to the cutting effect.
According to an embodiment of the present invention, the first graph cutting module 3 is specifically configured to: if the target frames are distributed evenly and the detected image contains all information of the target, adopting a window method map cutting strategy to cut the map; if the target frames are distributed evenly and the detected image contains partial information of the target, adopting a special position map cutting strategy; if the target frames are not distributed uniformly and the detected image contains all information of the target, adopting an edge-based map cutting strategy; and if the target frames are unevenly distributed and the detected image contains partial information of the target, adopting a clustering and graph cutting strategy.
According to the image cutting device based on target detection provided by the embodiment of the invention, an acquisition module acquires the position information and the size of a target frame in a detection image, and judges the distribution rule of the target frame according to the position information and the size of the target frame, a statistical module counts the non-black and non-white area and the pixel jumping condition in the detection image, and judges the distribution condition of a target in the detection image according to the non-black and non-white area and the pixel jumping condition, a first image cutting module selects an image cutting strategy according to the distribution rule of the target frame and the distribution condition of the target in the detection image, and cuts images according to the image cutting strategy, wherein the image cutting strategy comprises the following steps: the method comprises a sliding window method map cutting strategy, a special position map cutting strategy, a clustering map cutting strategy and an edge map cutting strategy, wherein a learning module inputs a map cut picture into a deep learning model for training to judge the map cutting effect, a second map cutting module selects Negative chips maps containing difficult Negative samples according to the map cutting effect, carries out region extraction on the Negative samples and carries out map cutting according to the extracted regions. Therefore, the device can select a proper image cutting strategy to automatically cut images according to the target, the position, the size and the distribution condition, and further evaluate the image cutting effect by combining with the subsequent deep learning model expression, thereby ensuring the image cutting effect and laying a foundation for the subsequent actual operation.
Furthermore, the present invention also proposes a non-transitory computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the above-mentioned object detection-based graph cutting method.
According to a non-transitory computer readable storage medium of an embodiment of the present invention, when executed by a processor, a computer program stored on a saddle is configured to acquire position information and size of a target frame in a detected image, determine a distribution rule of the target frame according to the position information and the size of the target frame, count non-black and non-white regions and pixel skipping situations in the detected image, determine a distribution situation of a target in the detected image according to the non-black and non-white regions and the pixel skipping situations, select a cropping policy according to the distribution rule of the target frame and the distribution situation of the target in the detected image, and perform cropping according to the cropping policy, wherein the cropping policy includes: the method comprises the steps of inputting a picture after picture cutting into a deep learning model for training to judge the picture cutting effect, selecting Negative chips containing difficult Negative samples according to the picture cutting effect, carrying out region extraction on the Negative samples, and cutting the picture according to the extracted region. Therefore, by selecting a proper map cutting strategy to carry out automatic map cutting according to the target, the position, the size and the distribution condition, and simultaneously, further evaluating the map cutting effect by combining with the subsequent deep learning model expression, the map cutting effect is ensured, and a foundation is laid for the subsequent actual operation.
In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.
Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.
The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.
It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.
In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.
The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
Claims (10)
1. A graph cutting method based on target detection is characterized by comprising the following steps:
acquiring position information and size of a target frame in a detection image, and judging a distribution rule of the target frame according to the position information and the size of the target frame;
counting the non-black non-white area and the pixel skipping condition in the detection image, and judging the distribution condition of the target in the detection image according to the non-black non-white area and the pixel skipping condition;
selecting a map cutting strategy according to the distribution rule of the target frame and the distribution condition of the target in the detection image, and cutting the map according to the map cutting strategy, wherein the map cutting strategy comprises the following steps: the method comprises the following steps of (1) a sliding window method map cutting strategy, a special position map cutting strategy, a clustering map cutting strategy and an edge-based map cutting strategy;
inputting the picture after the picture cutting into a deep learning model for training so as to judge the picture cutting effect;
selecting Negative chips containing difficult Negative samples according to the cutting effect, carrying out region extraction on the Negative samples, and cutting according to the extracted regions.
2. The target detection-based graph cutting method according to claim 1, further comprising:
acquiring the distribution rule change of a target frame of the picture after the picture is cut and the size change of the target frame;
inputting the image after the image cutting into a single-stage target detection model or a double-stage target detection model to obtain the algorithm expression of the image after the image cutting, wherein the algorithm expression comprises the following steps: the over-killing rate of the target, the omission factor of the target, the accuracy rate of the target, a confusion matrix of the target, a class AP value and an overall mAP value;
and correcting the graph cutting strategy according to the distribution rule change of the target frame, the size change of the target frame and the algorithm performance.
3. The target detection-based graph cutting method according to claim 2, wherein the single-stage target detection model comprises: YOLOv5 model, SSD model, the two-stage target detection model comprising: fast R-CNN, Cascade R-CNN and HTC.
4. The method as claimed in claim 1, wherein based on the mapping effect, Negative chips mapping containing hard Negative samples are selected based on SNIPER algorithm.
5. The method for cutting the graph based on the target detection according to claim 1, wherein selecting the graph cutting strategy according to the distribution rule of the target frame and the distribution condition of the target in the detection image comprises:
if the target frames are distributed evenly and the detection image contains all information of the target, a window method image cutting strategy is adopted for cutting the image;
if the target frames are distributed evenly and the detection image contains partial information of the target, adopting a special position map cutting strategy;
if the target frame is unevenly distributed and the detection image contains all information of the target, adopting an edge-based map cutting strategy;
and if the target frames are not distributed uniformly and the detection image contains partial information of the target, adopting a clustering and graph cutting strategy.
6. A map cutting device based on target detection is characterized by comprising:
the acquisition module is used for acquiring the position information and the size of a target frame in a detection image and judging the distribution rule of the target frame according to the position information and the size of the target frame;
the statistical module is used for counting the non-black non-white area and the pixel jumping condition in the detection image and judging the distribution condition of the target in the detection image according to the non-black non-white area and the pixel jumping condition;
the first map cutting module is used for selecting a map cutting strategy according to the distribution rule of the target frame and the distribution condition of the target in the detection image, and cutting the map according to the map cutting strategy, wherein the map cutting strategy comprises the following steps: the method comprises the following steps of (1) a sliding window method map cutting strategy, a special position map cutting strategy, a clustering map cutting strategy and an edge-based map cutting strategy;
the learning module is used for inputting the cut pictures into a deep learning model for training so as to judge the cut effect;
and the second map cutting module is used for picking out Negative chips containing difficult Negative samples according to the map cutting effect, carrying out region extraction on the Negative samples and carrying out map cutting according to the extracted regions.
7. The object detection-based graph cutting device according to claim 6, further comprising a modification module for:
acquiring the distribution rule change of a target frame of the picture after the picture is cut and the size change of the target frame;
inputting the image after the image cutting into a single-stage target detection model or a double-stage target detection model to obtain the algorithm expression of the image after the image cutting, wherein the algorithm expression comprises the following steps: the over-killing rate of the target, the omission factor of the target, the accuracy rate of the target, a confusion matrix of the target, a class AP value and an overall mAP value;
and correcting the graph cutting strategy according to the distribution rule change of the target frame, the size change of the target frame and the algorithm performance.
8. The target detection-based map cutting apparatus according to claim 6, wherein the second map cutting module is specifically configured to:
and selecting Negative chips containing hard Negative samples based on a SNIPER algorithm according to the cutting effect.
9. The object detection-based map cutting apparatus of claim 6, wherein the first map cutting module is specifically configured to:
if the target frames are distributed evenly and the detection image contains all information of the target, a window method image cutting strategy is adopted for cutting the image;
if the target frames are distributed evenly and the detection image contains partial information of the target, adopting a special position map cutting strategy;
if the target frame is unevenly distributed and the detection image contains all information of the target, adopting an edge-based map cutting strategy;
and if the target frames are not distributed uniformly and the detection image contains partial information of the target, adopting a clustering and graph cutting strategy.
10. A non-transitory computer-readable storage medium having stored thereon a computer program, wherein the program, when executed by a processor, implements the object detection-based graph cut method according to any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110248957.8A CN112927247A (en) | 2021-03-08 | 2021-03-08 | Graph cutting method based on target detection, graph cutting device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110248957.8A CN112927247A (en) | 2021-03-08 | 2021-03-08 | Graph cutting method based on target detection, graph cutting device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112927247A true CN112927247A (en) | 2021-06-08 |
Family
ID=76171790
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110248957.8A Pending CN112927247A (en) | 2021-03-08 | 2021-03-08 | Graph cutting method based on target detection, graph cutting device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112927247A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113870196A (en) * | 2021-09-10 | 2021-12-31 | 苏州浪潮智能科技有限公司 | Image processing method, device, equipment and medium based on anchor point cutting graph |
| CN114622311A (en) * | 2022-05-17 | 2022-06-14 | 北京东方国信科技股份有限公司 | Yarn breakage detection method and device and spinning machine |
| CN115496749A (en) * | 2022-11-14 | 2022-12-20 | 江苏智云天工科技有限公司 | Product defect detection method and system based on target detection training preprocessing |
| CN116167113A (en) * | 2023-04-19 | 2023-05-26 | 华联世纪工程咨询股份有限公司 | Automatic graph cutting method based on graph range |
-
2021
- 2021-03-08 CN CN202110248957.8A patent/CN112927247A/en active Pending
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113870196A (en) * | 2021-09-10 | 2021-12-31 | 苏州浪潮智能科技有限公司 | Image processing method, device, equipment and medium based on anchor point cutting graph |
| WO2023035558A1 (en) * | 2021-09-10 | 2023-03-16 | 苏州浪潮智能科技有限公司 | Anchor point cut-based image processing method and apparatus, device, and medium |
| CN114622311A (en) * | 2022-05-17 | 2022-06-14 | 北京东方国信科技股份有限公司 | Yarn breakage detection method and device and spinning machine |
| CN115496749A (en) * | 2022-11-14 | 2022-12-20 | 江苏智云天工科技有限公司 | Product defect detection method and system based on target detection training preprocessing |
| CN115496749B (en) * | 2022-11-14 | 2023-01-31 | 江苏智云天工科技有限公司 | Product defect detection method and system based on target detection training preprocessing |
| CN116167113A (en) * | 2023-04-19 | 2023-05-26 | 华联世纪工程咨询股份有限公司 | Automatic graph cutting method based on graph range |
| CN116167113B (en) * | 2023-04-19 | 2023-07-04 | 华联世纪工程咨询股份有限公司 | Automatic graph cutting method based on graph range |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112927247A (en) | Graph cutting method based on target detection, graph cutting device and storage medium | |
| US8837836B2 (en) | Image processing device identifying attribute of region included in image | |
| US10165248B2 (en) | Optimization method of image depth information and image processing apparatus | |
| US8660350B2 (en) | Image segmentation devices and methods based on sequential frame image of static scene | |
| CN105631418A (en) | People counting method and device | |
| CN113109368B (en) | Glass crack detection method, device, equipment and medium | |
| CN110233971B (en) | Shooting method, terminal and computer readable storage medium | |
| JP2010220197A (en) | Device and method for detecting shadow in image | |
| US10817744B2 (en) | Systems and methods for identifying salient images | |
| CN109948393A (en) | A kind of localization method and device of bar code | |
| CN111401290A (en) | Face detection method and system and computer readable storage medium | |
| CN110599453A (en) | Panel defect detection method and device based on image fusion and equipment terminal | |
| CN114782412A (en) | Image detection method, and training method and device of target detection model | |
| CN114511820A (en) | Goods shelf commodity detection method and device, computer equipment and storage medium | |
| CN109816720B (en) | Road center detection method, airborne equipment and storage medium | |
| CN113869230A (en) | Football goal type identification method, device, system and storage medium | |
| TW201911230A (en) | Surveillance method, computing device, and non-transitory storage medium | |
| CN114155241A (en) | Foreign matter detection method and device and electronic equipment | |
| CN106415596B (en) | image conversion based on segmentation | |
| CN110008792B (en) | Image detection method, image detection device, computer equipment and storage medium | |
| CN115861315B (en) | Defect detection method and device | |
| CN113256608A (en) | Workpiece defect detection method and device | |
| CN113516625A (en) | Method, device and equipment for detecting abnormity of photovoltaic module image | |
| WO2024016632A1 (en) | Bright spot location method, bright spot location apparatus, electronic device and storage medium | |
| WO2024011888A1 (en) | License plate recognition method and apparatus, and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |