CN117372966B - Turntable state monitoring method based on video monitoring - Google Patents

Turntable state monitoring method based on video monitoring Download PDF

Info

Publication number
CN117372966B
CN117372966B CN202311657186.3A CN202311657186A CN117372966B CN 117372966 B CN117372966 B CN 117372966B CN 202311657186 A CN202311657186 A CN 202311657186A CN 117372966 B CN117372966 B CN 117372966B
Authority
CN
China
Prior art keywords
target
turntable
pixel point
sequence
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311657186.3A
Other languages
Chinese (zh)
Other versions
CN117372966A (en
Inventor
韩大龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shaanxi Longyue Ruixing Technology Co ltd
Original Assignee
Shaanxi Longyue Ruixing Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shaanxi Longyue Ruixing Technology Co ltd filed Critical Shaanxi Longyue Ruixing Technology Co ltd
Priority to CN202311657186.3A priority Critical patent/CN117372966B/en
Publication of CN117372966A publication Critical patent/CN117372966A/en
Application granted granted Critical
Publication of CN117372966B publication Critical patent/CN117372966B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/52Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01PMEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT
    • G01P3/00Measuring linear or angular speed; Measuring differences of linear or angular speeds
    • G01P3/64Devices characterised by the determination of the time taken to traverse a fixed distance
    • G01P3/68Devices characterised by the determination of the time taken to traverse a fixed distance using optical means, i.e. using infrared, visible, or ultraviolet light
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/20Image enhancement or restoration by the use of local operators
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761Proximity, similarity or dissimilarity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/40Scenes; Scene-specific elements in video content
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20024Filtering details
    • G06T2207/20032Median filtering
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07Target detection

Abstract

The invention relates to the technical field of image processing, in particular to a turntable state monitoring method based on video monitoring, which comprises the following steps: acquiring gray images of a plurality of frames of rotary tables during operation; acquiring each target category region of the target image when each frame of turntable operates according to the gray distribution similarity between each pixel point in the gray image when each frame of turntable operates and each pixel point in the eight adjacent areas of the pixel point; acquiring linear speeds of all target category areas in a target image when the turntable operates; and acquiring the angular speed of the turntable during operation according to the linear speeds of all the target category areas in the target image during operation of the turntable, and further acquiring the operation state of the turntable. The angular speed of the turntable is not influenced by the external environment when the turntable runs, and the accuracy of the measurement result is higher.

Description

Turntable state monitoring method based on video monitoring
Technical Field
The invention relates to the technical field of image processing, in particular to a turntable state monitoring method based on video monitoring.
Background
Turntable monitoring technology is commonly used in a variety of industries including manufacturing, national defense, telecommunications, energy, construction, scientific research, and the like. The detection of the state of the turntable by video monitoring is a commonly used method at present, and is convenient for monitoring whether the running state of the turntable of the industrial equipment is normal or not. Status monitoring of industrial equipment turrets refers to the use of various sensors and monitoring technologies to monitor the status and performance of rotating components on industrial equipment in real time. This helps to ensure proper operation, safety and reliability of the apparatus, while also helping to improve production efficiency and reduce downtime. The state monitoring of the industrial equipment turntable is beneficial to early identification of equipment problems, avoids sudden faults, improves production efficiency, prolongs equipment life and reduces maintenance cost.
When the actual rotation speed of the turntable is different from the set speed range, the turntable state is abnormal, and therefore, it is necessary to monitor the turntable state by measuring the angular speed at which the turntable operates. At present, the rotating speed of the turntable is usually measured through a mechanical sensor, a photoelectric sensor or an electromagnetic sensor, but the mechanical sensor is in contact with the turntable, so that the turntable is easy to wear, the measuring result of the photoelectric sensor is easy to be influenced by ambient light, and the measuring result of the electromagnetic sensor is easy to be influenced by an ambient magnetic field, so that the accuracy of the measuring result is difficult to ensure.
Therefore, how to obtain the angular velocity of the turntable during operation, which is not affected by the external environment and has higher accuracy of the measurement results, is a problem to be solved.
Disclosure of Invention
In order to solve the above problems, the present invention provides a method for monitoring a state of a turntable based on video monitoring, the method comprising:
acquiring gray images of a plurality of frames of rotary tables during operation;
acquiring gray distribution similarity between each pixel point in a gray image and each pixel point in the eight adjacent areas of the pixel point when each frame of turntable operates;
taking a gray level image of any frame of turntable in operation as a target image of the turntable in operation, constructing an undirected graph of the target image of the turntable in operation according to gray level distribution similarity of each pixel point and each pixel point in eight adjacent areas of the pixel point, and clustering the undirected graph to obtain a plurality of category areas of the target image of the turntable in operation; acquiring each target category region of the target image when the turntable runs according to the centroid position of each category region;
acquiring a Gaussian mixture model curve of each pixel point in a target image when the turntable operates; acquiring all target gray scale intervals of each pixel point in each target class area according to the Gaussian mixture model curve of each pixel point; acquiring a target sequence number of each pixel point according to the sequence number of a target gray scale interval to which a gray scale value of a gray scale image belongs when each pixel point runs on each frame of turntable; acquiring all target pixel points of each target class area according to the number of the sequence numbers of each target gray scale interval in the target sequence number sequence of each pixel point in each target class area; acquiring a hole frame number interval value of each target pixel point according to the position distribution of each serial number type in the target serial number sequence of each target pixel point; acquiring a hole appearance time interval value of each target pixel point according to the hole frame number interval value of each target pixel point; acquiring the linear speeds of all target category areas in the target image when the turntable operates according to the hole occurrence time interval value of each target pixel point;
acquiring the angular velocity of the turntable during operation according to the linear velocities of all target category areas in the target image during operation of the turntable, and acquiring the operation state of the turntable according to the angular velocity of the turntable during operation;
the method for acquiring the Gaussian mixture model curve of each pixel point in the target image during turntable operation comprises the following steps: and carrying out Gaussian mixture background modeling on the gray level image of each frame of turntable during operation to obtain a Gaussian mixture model curve of each pixel point in the target image of each turntable during operation.
Preferably, the method for obtaining the gray distribution similarity between each pixel point in the gray image of each frame turntable during operation and each pixel point in the eight adjacent areas of the pixel point includes the following specific steps:
presetting a parameterCarrying out Gaussian mixture background modeling on the gray level image during operation of each frame of turntable to obtain a weight value, a mean value parameter and a standard deviation parameter of each sub Gaussian model of each pixel point in the gray level image during operation of each frame of turntable; for->The +.f in gray scale image when frame turntable is running>A pixel dot for adding->All sub-Gaussian models of each pixel point are changed from large to small according to the weight value of the sub-Gaussian modelOrdering to obtain->sub-Gaussian model sequence of each pixel point is +.>Front +.>A sub-Gaussian model as +.>A target sub-Gaussian model sequence of the individual pixel points; then->The +.f in gray scale image when frame turntable is running>A pixel point and the eighth pixel point in eight adjacent areas>The method for calculating the gray level distribution similarity of each pixel point comprises the following steps:
in the method, in the process of the invention,indicate->The +.f in gray scale image when frame turntable is running>A pixel point and the eighth pixel point in eight adjacent areas>Gray level distribution similarity of individual pixel points; />Representing acquisition->Maximum value of>Indicate->The +.f in gray scale image when frame turntable is running>First +.in target sub-Gaussian model sequence of each pixel point>Weight value of individual sub-Gaussian model, +.>Indicate->The +.f in gray scale image when frame turntable is running>Eighth pixel point eighth adjacent areaFirst +.in target sub-Gaussian model sequence of each pixel point>Weight values of the individual sub-gaussian models; />Indicate->The +.f in gray scale image when frame turntable is running>First +.in target sub-Gaussian model sequence of each pixel point>Average parameters of the individual gaussian models; />Indicate->The +.f in gray scale image when frame turntable is running>Eighth +.>First +.in target sub-Gaussian model sequence of each pixel point>Average parameters of the individual gaussian models; />Indicate->The +.f in gray scale image when frame turntable is running>First +.in target sub-Gaussian model sequence of each pixel point>Standard deviation parameters of the individual sub-gaussian models; />Indicate->The +.f in gray scale image when frame turntable is running>Eighth +.>Target of each pixel pointFirst>Standard deviation parameters of the individual sub-gaussian models; />The representation takes absolute value.
Preferably, the method for constructing the undirected graph of the turntable operation target image according to the gray distribution similarity between each pixel point and each pixel point in the eight adjacent areas of the pixel point, and clustering the undirected graph to obtain a plurality of category areas of the turntable operation target image includes the following specific steps:
taking the gray distribution similarity of each pixel point in the target image when the turntable operates and each pixel point in the eight adjacent areas of the pixel points as an edge weight value of the undirected graph, and constructing the undirected graph of the target image when the turntable operates; and performing spectral clustering operation on the undirected graph of the target image during turntable operation to obtain a plurality of category areas of the target image during turntable operation.
Preferably, the method for obtaining all the target gray scale intervals of each pixel point in each target class area according to the mixed gaussian model curve of each pixel point includes the following specific steps:
the first image of the object when the turntable runs by peak detection methodThe>Peak detection is carried out on the Gaussian mixture model curve of each pixel point, and the +.>All minima of the Gaussian mixture model curve of each pixel point; the gray value range between the first minimum value and the second minimum value is taken as the +.>A first target gray scale interval of each pixel point, a second target gray scale interval is extremely smallGray value range between the value and the third minimum value as +.>A second target gray scale interval of the pixel points takes a gray scale value range between a third minimum value and a fourth minimum value as a first gray scale interval +.>A third target gray scale interval of the pixel points; by analogy, get +.>All target gray scale intervals of the pixel points.
Preferably, the method for obtaining all the target pixel points in each target class area according to the number of the sequence numbers of each target gray scale interval in the sequence of the target sequence numbers of each pixel point in each target class area includes the following specific steps:
counting the number of sequence numbers in each target gray scale interval in a target sequence number sequence corresponding to each pixel point and the total number of all sequence numbers of the target sequence number sequence;
presetting a parameterIf the first +.>The>The number and +.>The ratio of the total number of all sequence numbers of the target sequence number sequence of each pixel point is smaller than +.>Will be->The pixel points are marked as target pixel points;
wherein, at the firstIn all target gray scale intervals of each pixel point, the first part is acquired>The sequence of the target gray scale interval to which the gray scale value of the gray scale image belongs when each pixel point runs on each frame of turntable is marked as the +.>Target sequence number sequence of individual pixels.
Preferably, the obtaining the hole frame number interval value of each target pixel point according to the position distribution of each serial number category in the target serial number sequence of each target pixel point includes the following specific methods:
marking any target sequence number sequence of any target pixel point in any target class area of the target image during turntable operation as a b-th sequence number class and marking the sequence number sequence as a b-th sequence number class of the target image during turntable operationThe>The first part in the target sequence number sequence of each target pixel point>The serial number category will appear successively +.>The sequence number segment composed of the sequence numbers of the sequence number categories is marked as +.>Target sequence number segment of sequence number category, and further obtain +.>All target sequence number segments of the sequence number class; the number of the sequence numbers between any two adjacent target sequence number segments is marked as +.>Discrete interval values of sequence number categories, thereby obtaining +.>All discontinuous interval values of the sequence number classes; will be->Mean value of all discontinuous interval values of the sequence number category +.>Normalized value of product of number of all target sequence number segments of sequence number category as +.>The likelihood of holes of the sequence number variety; thereby obtaining->The possibility of holes of each serial number type in the target serial number sequence of each target pixel point is +.>The serial number type with the highest possibility of holes in the target serial number sequence of each target pixel is taken as the +.>The target sequence number of the target sequence number sequence of each target pixel point is the average value of all discontinuous interval values of the target sequence number sequence as the +.>And the hole frame number interval value of each target pixel point.
Preferably, the specific formula for obtaining the hole appearance time interval value of each target pixel point according to the hole frame number interval value of each target pixel point is as follows:
in the method, in the process of the invention,representing the target image at turntable operation +.>The>The hole appearance time interval values of the target pixel points; />The +.o. representing the target image when the turntable is running>The>Hole frame number interval values of the target pixel points; />Representing a two frame interval.
Preferably, the line speed of all target category areas in the target image when the turntable operates is obtained according to the hole occurrence time interval value of each target pixel point, and the specific method comprises the following steps:
the distance parameter between the adjacent holes of the turntable and the first target image of the turntable during operationThe ratio of the average value of the hole occurrence time interval values of all the target pixel points in the target category areas is used as the +.>Linear velocity of each target class area.
Preferably, the obtaining the angular velocity of the turntable during operation according to the linear velocities of all target category areas in the target image during turntable operation includes the following specific methods:
and taking the ratio of the average value of the linear speeds of all the target category areas in the target image when the turntable operates to the radius of the turntable as the angular speed when the turntable operates.
Preferably, the method for acquiring each target category region of the target image during the running of the turntable according to the centroid position of each category region includes the following specific steps:
and acquiring the centroid position of each class area of the turntable operation time target image through a pixel level centroid method for each class area of the turntable operation time target image, and taking the class area with the same centroid position and the largest number of class areas in the turntable operation time target image as each target class area of the turntable operation time target image.
The technical scheme of the invention has the beneficial effects that: dividing an image by calculating the gray distribution similarity of each pixel point in each frame of image and each pixel point in eight adjacent areas of the image to obtain a plurality of target category areas, and obtaining a hole occurrence time interval value of each target pixel point according to a hole frame number interval value of each target pixel point in the target category areas; according to the hole occurrence time interval value of each target pixel point, acquiring the linear speeds of all target category areas in the target image when the turntable operates, and further acquiring the angular speed when the turntable operates; according to the invention, the angular speed of the turntable is not influenced by external environment, the accuracy of the measurement result is higher, and the running state of the turntable is monitored by the accurate angular speed of the turntable.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of the steps of the method for monitoring the state of a turntable based on video monitoring.
Detailed Description
In order to further describe the technical means and effects adopted by the invention to achieve the preset aim, the following detailed description is given below of the specific implementation, structure, characteristics and effects of the turntable state monitoring method based on video monitoring according to the invention with reference to the attached drawings and the preferred embodiment. In the following description, different "one embodiment" or "another embodiment" means that the embodiments are not necessarily the same. Furthermore, the particular features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The following specifically describes a specific scheme of the turntable state monitoring method based on video monitoring provided by the invention with reference to the accompanying drawings.
Referring to fig. 1, a flowchart illustrating steps of a method for monitoring a status of a turntable based on video monitoring according to an embodiment of the present invention is shown, the method includes the following steps:
step S001: and acquiring gray images of a plurality of frames of turntable in operation.
When the turntable set speed is different from the actual rotation speed, the turntable state may be abnormal. At present, the rotating speed of the turntable is usually measured through a mechanical sensor, a photoelectric sensor or an electromagnetic sensor, but the mechanical sensor is in contact with the turntable, so that the turntable is easy to wear, the photoelectric sensor is easy to be influenced by ambient light, the electromagnetic sensor is easy to be influenced by an ambient magnetic field, the accuracy of the measuring result is difficult to be ensured, and the follow-up step operation is carried out according to the defects of the existing sensor.
Specifically, in order to implement the turntable state monitoring method based on video monitoring provided in this embodiment, firstly, gray images during running of a plurality of frames of turntable need to be collected, and the specific process is as follows:
when the turntable is at rest, the minimum boundary distance between any two adjacent holes in the turntable is measured by using a graduated scale, and the minimum boundary distance is used as a distance parameter between the adjacent holes of the turntable, and the radius of the turntable is measured and acquired by using the graduated scale.
The method comprises the steps that a high-resolution industrial camera is adopted to capture video images when the turntable runs, the industrial camera equipment is connected with video signal transmission equipment to transmit the video transmission images to a monitoring system terminal, and the terminal is a local monitoring station and is used for receiving, storing and analyzing the video images acquired by monitoring when the turntable runs; dividing the video image during turntable operation into a plurality of frame turntable operation images according to the frame rate of the video, and carrying out median filtering denoising and graying operation on the plurality of frame turntable operation images to obtain a plurality of frame turntable operation gray images. The median filtering and graying operation is in the prior art, and is not repeated here, the time intervals of any two adjacent frames are the same, and the minimum boundary distances between any two adjacent holes in the turntable are the same.
So far, the gray level images of a plurality of frames of rotary tables in operation are obtained through the method.
Step S002: and acquiring each target category region of the target image when each frame of turntable operates according to the gray distribution similarity between each pixel point in the gray image when each frame of turntable operates and each pixel point in the eight adjacent areas of the pixel point.
1. And acquiring the gray distribution similarity between each pixel point in the gray image and each pixel point in the eight adjacent areas of the pixel point in each frame of turntable operation.
It should be noted that, the gray level image is subjected to gaussian background modeling when a plurality of frames of turntable run, and a mixed gaussian model of pixel points at the same position in all frames of images is output, so that parameters of all sub gaussian models of the mixed gaussian model are obtained. Because the Gaussian mixture models at different positions have different characteristics, when the positions are at the small hole positions of the turntable area, two obvious peaks appear on the Gaussian mixture model curve, one is a small hole part, and the other is a metal part between the small holes; and when the position is a background area or other pure metal areas, only one peak may appear in the mixed Gaussian model curve. The gray values of the pixel points at the same position in each frame image are different due to the operation of the turntable, the Gaussian mixture model constructed based on the gray values of the pixel points at the same position in each frame image shows the change, and the pixel points at each adjacent position possibly belong to the same category, so that the embodiment divides each part in the image to obtain a plurality of categories by calculating the gray distribution similarity of each pixel point in each frame image and each pixel point in eight adjacent positions.
Presetting a parameterWherein the present embodiment is +.>To describe the example, the present embodiment is not particularly limited, wherein +.>Depending on the particular implementation.
Specifically, carrying out Gaussian mixture background modeling on the gray level image during operation of each frame of turntable, and obtaining a weight value, a mean value parameter and a standard deviation parameter of each sub Gaussian model of each pixel point in the gray level image during operation of each frame of turntable; for the firstThe +.f in gray scale image when frame turntable is running>A pixel dot for adding->All sub-Gaussian models of each pixel point are sequenced from big to small according to the weight values of the sub-Gaussian models, and the +.>sub-Gaussian model sequence of each pixel point is +.>Front +.>A sub-Gaussian model as +.>A target sub-Gaussian model sequence of the individual pixel points; then->The +.f in gray scale image when frame turntable is running>A pixel point and the eighth pixel point in eight adjacent areas>The method for calculating the gray level distribution similarity of each pixel point comprises the following steps:
in the method, in the process of the invention,indicate->The +.f in gray scale image when frame turntable is running>A pixel point and the eighth pixel point in eight adjacent areas>Gray level distribution similarity of individual pixel points; />Indicate->The +.f in gray scale image when frame turntable is running>The number of all sub-gaussian models of the target sub-gaussian model sequence of the individual pixels; />Representing acquisition->Maximum value of>Indicate->The +.f in gray scale image when frame turntable is running>First +.in target sub-Gaussian model sequence of each pixel point>Weight value of individual sub-Gaussian model, +.>Indicate->The +.f in gray scale image when frame turntable is running>Eighth +.>First +.in target sub-Gaussian model sequence of each pixel point>Weight values of the individual sub-gaussian models; />Indicate->The +.f in gray scale image when frame turntable is running>First +.in target sub-Gaussian model sequence of each pixel point>Average parameters of the individual gaussian models;indicate->The +.f in gray scale image when frame turntable is running>Eighth +.>First +.in target sub-Gaussian model sequence of each pixel point>Average parameters of the individual gaussian models; />Indicate->The +.f in gray scale image when frame turntable is running>First +.in target sub-Gaussian model sequence of each pixel point>Standard deviation parameters of the individual sub-gaussian models; />Indicate->The +.f in gray scale image when frame turntable is running>Eighth +.>First +.in target sub-Gaussian model sequence of each pixel point>Standard deviation parameters of the individual sub-gaussian models; />The representation takes absolute value.
It should be noted that, in the prior art, the gray level image is modeled in a gaussian background manner when each frame of turntable is running, and the weights of all sub-gaussian models in the mixed gaussian model reflect the distribution condition of gray level values of pixels at the same position in different frames under all sub-gaussian models, when the weights of the sub-gaussian models are larger, the gray level distribution of the pixels at the position in the frame with more descriptions accords with the distribution of the sub-gaussian model, and when the weights of the sub-gaussian models are smaller, the gray level distribution of the pixels at the position in the frame with less descriptions accords with the distribution of the sub-gaussian model. Therefore, when the similarity of gray distribution of the pixels at two adjacent positions in all frames is obtained, the difference between the sub-gaussian models with larger weights in the mixture gaussian models at the two adjacent positions should be more focused, and when the difference between the sub-gaussian models with larger weights is smaller, the gray distribution of the pixels at the two adjacent positions in more frames is more similar, and the pixels at the two adjacent positions are more likely to represent the same image feature. Therefore, in this embodiment, the weight of the sub-gaussian model in the mixed gaussian model is used as the weight between the differences of the sub-gaussian models in two positions, the greater the weight is, the more attention is paid to the differences of the sub-gaussian models, and the less attention is paid to the differences of the sub-gaussian models with smaller weights.
So far, the gray distribution similarity between each pixel point in the gray image and each pixel point in the eight adjacent areas of the pixel point in each frame of turntable operation is obtained.
And taking the gray level image of any frame of turntable in running as a target image of turntable in running.
2. Each target class area of the turret run-time target image is acquired.
The image comprises a turntable area and a background area, and the edge weight of the undirected graph is determined according to the gray level distribution similarity of each pixel point and each pixel point in the eight adjacent areas of the image, so that the image is conveniently divided into the turntable area and the background area by taking the edge weight as the basis for performing undirected graph clustering operation; through spectral clustering, continuous pixel points with very large similarity (namely similar gray distribution) of the Gaussian mixture model can be clustered into one type. And because there are multiple category parts in the image, such as a turntable, a background, etc. The similarity of the Gaussian mixture models of different classes is different, so that the final spectral clustering results are different. In particular, for each category in the turntable region, the whole is caused to take the form of concentric circles, so that the positions of the mass centers of the whole are consistent, and the characteristics which are not possessed by each category of the non-turntable region are not possessed.
Specifically, the gray distribution similarity of each pixel point in the target image when the turntable operates and each pixel point in the eight adjacent areas of the pixel points is used as the edge weight value of the undirected graph, and the undirected graph of the target image when the turntable operates is constructed; and performing spectral clustering operation on the undirected graph of the target image during turntable operation to obtain a plurality of category areas of the target image during turntable operation.
The undirected graph and the spectral clustering are constructed in the prior art, and are not repeated here.
It should be noted that in the turntable area, each small hole forms a plurality of concentric circles with different sizes, the centers of mass in each circle are at the same point, and other background areas do not exist, so that we can acquire the turntable area in the image; the black holes are distributed in a circumference, the characteristics of the black holes are different from those of the metal turntable, the corresponding Gaussian mixture models are different, the circumferences of the black holes can be separated from the circumferences only comprising the metal turntable through clustering, but the circle centers of the circumferences are the same, namely the mass centers of a plurality of categories of turntable areas are basically consistent. For the pixel points in the background, the mass centers of the pixel points are greatly different from the mass centers of the turntable area after clustering, so that the embodiment screens all the categories of the turntable area through the distribution of the mass centers of all the categories.
Specifically, the centroid position of each class area of the object image in the turntable operation is obtained through a pixel level centroid method for each class area of the object image in the turntable operation, and the class area which has the class area with the same centroid position and has the largest class area number in the object image in the turntable operation is used as each object class area of the object image in the turntable operation.
The pixel level centroid method is in the prior art, and the description of this embodiment is not repeated.
So far, each target category area of the target image of the turntable in operation is obtained through the method.
Step S003: and acquiring the linear speeds of all target category areas in the target image when the turntable runs.
It should be noted that, through the above process, we can obtain each target class area of the target image during turntable operation, analyze the distribution condition of the gray image during multi-frame turntable operation of the pixel value at the same position in each target class area under the mixed gaussian model thereof, so as to determine the occurrence time interval of two adjacent holes in the gray image during turntable operation, and calculate the linear velocity of each target class area in the gray image during turntable operation through the hole occurrence time interval value of the pixel point in each target class area of the target image during turntable operation.
Specifically, carrying out Gaussian mixture background modeling on the gray level image of each frame of turntable during operation to obtain a Gaussian mixture model curve of each pixel point in the target image of each turntable during operation; for the first of the turntable run-time target imagesTarget class areaDomain->A pixel dot, the->The horizontal axis of the Gaussian mixture model curve of each pixel point is +.>The gray value of the target image when the turntable runs is displayed on each pixel point, and the vertical axis is the frequency of each gray value.
By peak detection methodPeak detection is carried out on the Gaussian mixture model curve of each pixel point, and the +.>All minima of the Gaussian mixture model curve of each pixel point; the gray value range between the first minimum value and the second minimum value is taken as the +.>A first target gray scale interval of each pixel point takes a gray scale value range between a second minimum value and a third minimum value as a first gray scale interval +.>A second target gray scale interval of the pixel points takes a gray scale value range between a third minimum value and a fourth minimum value as a first gray scale interval +.>A third target gray scale interval of the pixel points; by analogy, get +.>All target gray scale intervals of the pixel points.
It should be noted that, filtering all the obtained target gray scale intervals of each pixel, and using the distribution characteristics of the sub-gaussian model to calculate the time interval of the adjacent holes in the current target class area, what is needed is a target class area including black holes. The target class area also comprises a class without black holes, the target class area is completely composed of metal, and the composition ratio of the corresponding sequence of the target class area is preset as a preset parameter A. Therefore, the identical sequences in the obtained sequences under each target class area need to be removed, so that the accuracy of the subsequent data acquisition is ensured.
Presetting a parameterWherein the present embodiment is +.>To describe the example, the present embodiment is not particularly limited, wherein +.>Depending on the particular implementation.
Specifically, at the firstIn all target gray scale intervals of each pixel point, the first part is acquired>The sequence of the target gray scale interval to which the gray scale value of the gray scale image belongs when each pixel point runs on each frame of turntable is marked as the +.>And counting the number of the sequence numbers in each target gray scale interval in the target sequence number sequence corresponding to each pixel point and the total number of all the sequence numbers of the target sequence number sequence. In the embodiment of the invention, the number of the sequence numbers in each target gray scale interval in the target sequence number sequence corresponding to each pixel point and the total number of all the sequence numbers of the target sequence number are obtained by a statistical method, and in other embodiments, each image can be obtained directly by counting one by oneThe number of sequence numbers in each target gray scale interval in the target sequence number sequence corresponding to the pixel point and the total number of all sequence numbers of the target sequence number sequence; if at->The number and +.>The ratio of the total number of all sequence numbers of the target sequence number sequence of each pixel point is smaller than +.>Will be->The pixel is marked as the target pixel if at the firstThe number and the +.f. of any one of the sequence numbers of the target gray scale intervals in the target sequence number sequence of each pixel point>The ratio of the total number of all sequence numbers of the target sequence number sequence of each pixel point is greater than or equal to +.>Will be->The pixel points are marked as non-target pixel points; thereby obtaining->All target pixels of the target class area. Wherein->The target sequence number sequence of each pixel point comprises a plurality of sequence number types, and each sequence number type comprises a plurality of sequence numbers of target gray scale intervals.
Will be in any target class areaAny target sequence number sequence of any target pixel point is marked as the firstThe>The first part in the target sequence number sequence of each target pixel point>The serial number category will appear successively +.>The sequence number segment composed of the sequence numbers of the sequence number categories is marked as +.>Target sequence number segment of sequence number category, and further obtain +.>All target sequence number segments of the sequence number class; the number of the sequence numbers between any two adjacent target sequence number segments is marked as +.>Discrete interval values of sequence number categories, thereby obtaining +.>All discontinuous interval values of the sequence number classes; will be->Mean value of all discontinuous interval values of the sequence number category +.>Normalized value of product of number of all target sequence number segments of sequence number category as +.>The likelihood of holes of the sequence number variety; the average value of all discontinuous interval values is used for distinguishing background areas of a turntableAnd holes because the spacing between two holes is greater than the spacing between two turret backgrounds; the likelihood of the holes is calculated by the number of the target sequence number segments to prevent noise, and the fewer the noise types, the more holes. The greater the value of all the discrete intervals and the greater the number of all the target sequence number segments, the greater the likelihood of holes in the sequence number class. The likelihood of a hole is obtained by a normalized value of the product of the average value of the discontinuous interval values and the number of all target sequence segments, which is to represent that the average value of the discontinuous interval values and the number of all target sequence segments have positive correlation with the likelihood of the hole. Thereby obtaining->The possibility of holes of each serial number type in the target serial number sequence of each target pixel point is +.>The serial number type with the highest possibility of holes in the target serial number sequence of each target pixel is taken as the +.>The target sequence number of the target sequence number sequence of each target pixel point is the average value of all discontinuous interval values of the target sequence number sequence as the +.>And the hole frame number interval value of each target pixel point.
Further, the first object image of the turntable in runningThe>The calculation method of the hole occurrence time interval value of each target pixel point comprises the following steps:
in the method, in the process of the invention,representing the target image at turntable operation +.>The>The hole appearance time interval values of the target pixel points; />The +.o. representing the target image when the turntable is running>The>Hole frame number interval values of the target pixel points; />Representing a two frame interval.
Specifically, the distance parameter between the adjacent holes of the turntable and the first target image of the turntable during operationThe ratio of the average value of the hole occurrence time interval values of all the target pixel points in the target category areas is used as the +.>Linear velocities of the individual target class regions; and further obtaining the linear speeds of all target category areas in the target image during turntable operation.
So far, the linear speeds of all the target category areas in the target image when the turntable operates are obtained through the method.
Step S004: and acquiring the angular speed of the turntable during operation according to the linear speeds of all the target category areas in the target image during operation of the turntable, and further acquiring the operation state of the turntable.
The angular speed of the turntable during operation is compared with a preset standard angular speed range of the turntable when leaving the factory, so that whether the current operation state of the turntable is abnormal or not is determined.
Specifically, the ratio of the average value of the linear speeds of all target category areas in the target image when the turntable operates to the radius of the turntable is used as the angular speed when the turntable operates; if the angular speed of the turntable during operation is not in the preset standard angular speed range when the turntable leaves a factory, the operation state of the turntable is abnormal.
This embodiment is completed.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the invention, but any modifications, equivalent substitutions, improvements, etc. within the principles of the present invention should be included in the scope of the present invention.

Claims (9)

1. The turntable state monitoring method based on video monitoring is characterized by comprising the following steps of:
acquiring gray images of a plurality of frames of rotary tables during operation;
acquiring gray distribution similarity between each pixel point in a gray image and each pixel point in the eight adjacent areas of the pixel point when each frame of turntable operates;
taking a gray level image of any frame of turntable in operation as a target image of the turntable in operation, constructing an undirected graph of the target image of the turntable in operation according to gray level distribution similarity of each pixel point and each pixel point in eight adjacent areas of the pixel point, and clustering the undirected graph to obtain a plurality of category areas of the target image of the turntable in operation; acquiring each target category region of the target image when the turntable runs according to the centroid position of each category region;
acquiring a Gaussian mixture model curve of each pixel point in a target image when the turntable operates; acquiring all target gray scale intervals of each pixel point in each target class area according to the Gaussian mixture model curve of each pixel point; acquiring a target sequence number of each pixel point according to the sequence number of a target gray scale interval to which a gray scale value of a gray scale image belongs when each pixel point runs on each frame of turntable; acquiring all target pixel points of each target class area according to the number of the sequence numbers of each target gray scale interval in the target sequence number sequence of each pixel point in each target class area; acquiring a hole frame number interval value of each target pixel point according to the position distribution of each serial number type in the target serial number sequence of each target pixel point; acquiring a hole appearance time interval value of each target pixel point according to the hole frame number interval value of each target pixel point; acquiring the linear speeds of all target category areas in the target image when the turntable operates according to the hole occurrence time interval value of each target pixel point;
acquiring the angular velocity of the turntable during operation according to the linear velocities of all target category areas in the target image during operation of the turntable, and acquiring the operation state of the turntable according to the angular velocity of the turntable during operation;
the method for acquiring the Gaussian mixture model curve of each pixel point in the target image during turntable operation comprises the following steps: carrying out Gaussian mixture background modeling on the gray level image of each frame of turntable during operation to obtain a Gaussian mixture model curve of each pixel point in the target image of each turntable during operation;
according to the position distribution of each serial number category in the target serial number sequence of each target pixel point, the hole frame number interval value of each target pixel point is obtained, and the specific method comprises the following steps:
marking any target sequence number sequence of any target pixel point in any target class area of the target image during turntable operation as a b-th sequence number class and marking the sequence number sequence as a b-th sequence number class of the target image during turntable operationThe>The first part in the target sequence number sequence of each target pixel point>The number of serial number categories will appear continuously/>The sequence number segment composed of the sequence numbers of the sequence number categories is marked as +.>Target sequence number segment of sequence number category, and further obtain +.>All target sequence number segments of the sequence number class; the number of the sequence numbers between any two adjacent target sequence number segments is marked as +.>Discrete interval values of sequence number categories, thereby obtaining +.>All discontinuous interval values of the sequence number classes; will be->Mean value of all discontinuous interval values of the sequence number category +.>Normalized value of product of number of all target sequence number segments of sequence number category as +.>The likelihood of holes of the sequence number variety; thereby obtaining->The possibility of holes of each serial number type in the target serial number sequence of each target pixel point is +.>The serial number type with the highest possibility of holes in the target serial number sequence of each target pixel is taken as the +.>The target sequence number of the target sequence number sequence of each target pixel point is the average value of all discontinuous interval values of the target sequence number sequence as the +.>And the hole frame number interval value of each target pixel point.
2. The method for monitoring the state of a turntable based on video monitoring according to claim 1, wherein the specific method for obtaining the gray distribution similarity between each pixel point in the gray image of each frame of turntable operation and each pixel point in the eight neighboring areas of the pixel point comprises the following steps:
presetting a parameterCarrying out Gaussian mixture background modeling on the gray level image during operation of each frame of turntable to obtain a weight value, a mean value parameter and a standard deviation parameter of each sub Gaussian model of each pixel point in the gray level image during operation of each frame of turntable; for->The +.f in gray scale image when frame turntable is running>A pixel dot for adding->All sub-Gaussian models of each pixel point are sequenced from big to small according to the weight values of the sub-Gaussian models, and the +.>sub-Gaussian model sequence of each pixel point is +.>Of individual pixelsFront ∈in the sequence of the sub-Gaussian model>A sub-Gaussian model as +.>A target sub-Gaussian model sequence of the individual pixel points; then->The +.f in gray scale image when frame turntable is running>A pixel point and the eighth pixel point in eight adjacent areas>The method for calculating the gray level distribution similarity of each pixel point comprises the following steps:
in the method, in the process of the invention,indicate->The +.f in gray scale image when frame turntable is running>A pixel point and the eighth pixel point in eight adjacent areas>Gray level distribution similarity of individual pixel points; />Representing acquisition->Is selected from the group consisting of a maximum value of (c),indicate->The +.f in gray scale image when frame turntable is running>First +.in target sub-Gaussian model sequence of each pixel point>Weight value of individual sub-Gaussian model, +.>Indicate->The +.f in gray scale image when frame turntable is running>Eighth +.>First +.in target sub-Gaussian model sequence of each pixel point>Weight values of the individual sub-gaussian models; />Indicate->The +.f in gray scale image when frame turntable is running>In a target sub-Gaussian model sequence of each pixel pointFirst->Average parameters of the individual gaussian models; />Indicate->The +.f in gray scale image when frame turntable is running>Eighth +.>First +.in target sub-Gaussian model sequence of each pixel point>Average parameters of the individual gaussian models; />Indicate->The +.f in gray scale image when frame turntable is running>First +.in target sub-Gaussian model sequence of each pixel point>Standard deviation parameters of the individual sub-gaussian models; />Indicate->The +.f in gray scale image when frame turntable is running>Eighth +.>First +.in target sub-Gaussian model sequence of each pixel point>Standard deviation parameters of the individual sub-gaussian models; />The representation takes absolute value.
3. The method for monitoring the state of a turntable based on video monitoring according to claim 1, wherein the constructing an undirected graph of the target image during the turntable operation according to the gray level distribution similarity between each pixel point and each pixel point in the eight adjacent areas of the pixel point, and clustering the undirected graph to obtain a plurality of category areas of the target image during the turntable operation comprises the following specific methods:
taking the gray distribution similarity of each pixel point in the target image when the turntable operates and each pixel point in the eight adjacent areas of the pixel points as an edge weight value of the undirected graph, and constructing the undirected graph of the target image when the turntable operates; and performing spectral clustering operation on the undirected graph of the target image during turntable operation to obtain a plurality of category areas of the target image during turntable operation.
4. The method for monitoring the state of a turntable based on video monitoring according to claim 1, wherein the specific method for obtaining all the target gray intervals of each pixel point in each target class area according to the mixed gaussian model curve of each pixel point comprises the following steps:
the first image of the object when the turntable runs by peak detection methodThe>Peak detection is carried out on the Gaussian mixture model curve of each pixel point, and the +.>All minima of the Gaussian mixture model curve of each pixel point; the gray value range between the first minimum value and the second minimum value is taken as the +.>A first target gray scale interval of each pixel point takes a gray scale value range between a second minimum value and a third minimum value as a first gray scale interval +.>A second target gray scale interval of the pixel points takes a gray scale value range between a third minimum value and a fourth minimum value as a first gray scale interval +.>A third target gray scale interval of the pixel points; by analogy, get +.>All target gray scale intervals of the pixel points.
5. The method for monitoring the status of a turntable based on video monitoring according to claim 1, wherein the specific method for obtaining all the target pixels in each target class area according to the number of the sequence numbers of each target gray scale interval in the target sequence number sequence of each pixel in each target class area comprises the following steps:
counting the number of sequence numbers in each target gray scale interval in a target sequence number sequence corresponding to each pixel point and the total number of all sequence numbers of the target sequence number sequence;
presetting a parameterIf the first +.>The>The number and +.>The ratio of the total number of all sequence numbers of the target sequence number sequence of each pixel point is smaller than +.>Will be->The pixel points are marked as target pixel points;
wherein, at the firstIn all target gray scale intervals of each pixel point, the first part is acquired>The sequence of the target gray scale interval to which the gray scale value of the gray scale image belongs when each pixel point runs on each frame of turntable is marked as the +.>Target sequence number sequence of individual pixels.
6. The method for monitoring the state of a turntable based on video monitoring according to claim 1, wherein the specific formula for obtaining the hole appearance time interval value of each target pixel point according to the hole frame number interval value of each target pixel point is as follows:
in the method, in the process of the invention,representing the target image at turntable operation +.>The>The hole appearance time interval values of the target pixel points; />The +.o. representing the target image when the turntable is running>The>Hole frame number interval values of the target pixel points; />Representing a two frame interval.
7. The method for monitoring the state of a turntable based on video monitoring according to claim 1, wherein the method for obtaining the linear speeds of all target category areas in the target image of the turntable during operation according to the hole occurrence time interval value of each target pixel point comprises the following specific steps:
the distance parameter between the adjacent holes of the turntable and the first target image of the turntable during operationThe ratio of the average value of the hole occurrence time interval values of all the target pixel points in each target category area is used as a target when the turntable operatesImage +.>Linear velocity of each target class area.
8. The method for monitoring the state of the turntable based on video monitoring according to claim 1, wherein the method for obtaining the angular velocity of the turntable during operation according to the linear velocities of all target category areas in the target image during operation of the turntable comprises the following specific steps:
and taking the ratio of the average value of the linear speeds of all the target category areas in the target image when the turntable operates to the radius of the turntable as the angular speed when the turntable operates.
9. The method for monitoring the status of a turntable based on video monitoring according to claim 1, wherein the method for acquiring each target category region of the target image of the turntable during operation according to the centroid position of each category region comprises the following specific steps:
and acquiring the centroid position of each class area of the turntable operation time target image through a pixel level centroid method for each class area of the turntable operation time target image, and taking the class area with the same centroid position and the largest number of class areas in the turntable operation time target image as each target class area of the turntable operation time target image.
CN202311657186.3A 2023-12-06 2023-12-06 Turntable state monitoring method based on video monitoring Active CN117372966B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311657186.3A CN117372966B (en) 2023-12-06 2023-12-06 Turntable state monitoring method based on video monitoring

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311657186.3A CN117372966B (en) 2023-12-06 2023-12-06 Turntable state monitoring method based on video monitoring

Publications (2)

Publication Number Publication Date
CN117372966A CN117372966A (en) 2024-01-09
CN117372966B true CN117372966B (en) 2024-03-01

Family

ID=89398778

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311657186.3A Active CN117372966B (en) 2023-12-06 2023-12-06 Turntable state monitoring method based on video monitoring

Country Status (1)

Country Link
CN (1) CN117372966B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117692649B (en) * 2024-02-02 2024-04-19 广州中海电信有限公司 Ship remote monitoring video efficient transmission method based on image feature matching

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018130016A1 (en) * 2017-01-10 2018-07-19 哈尔滨工业大学深圳研究生院 Parking detection method and device based on monitoring video
JP2019028991A (en) * 2017-07-26 2019-02-21 富士通株式会社 Target detection method, target detection apparatus, and image processing apparatus
CN115272259A (en) * 2022-08-04 2022-11-01 苏州翰之辉电子科技有限公司 Method and system for detecting stability of running state of rotary table
CN115914649A (en) * 2023-03-01 2023-04-04 广州高通影像技术有限公司 Data transmission method and system for medical video
CN117132510A (en) * 2023-10-24 2023-11-28 临沂安保服务集团有限公司 Monitoring image enhancement method and system based on image processing

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7620266B2 (en) * 2005-01-20 2009-11-17 International Business Machines Corporation Robust and efficient foreground analysis for real-time video surveillance
US7664329B2 (en) * 2006-03-02 2010-02-16 Honeywell International Inc. Block-based Gaussian mixture model video motion detection
CN109543587B (en) * 2018-11-16 2020-04-24 中南大学 Method and system for detecting flow velocity of high-temperature molten fluid
US20230289979A1 (en) * 2020-11-13 2023-09-14 Zhejiang University A method for video moving object detection based on relative statistical characteristics of image pixels

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018130016A1 (en) * 2017-01-10 2018-07-19 哈尔滨工业大学深圳研究生院 Parking detection method and device based on monitoring video
JP2019028991A (en) * 2017-07-26 2019-02-21 富士通株式会社 Target detection method, target detection apparatus, and image processing apparatus
CN115272259A (en) * 2022-08-04 2022-11-01 苏州翰之辉电子科技有限公司 Method and system for detecting stability of running state of rotary table
CN115914649A (en) * 2023-03-01 2023-04-04 广州高通影像技术有限公司 Data transmission method and system for medical video
CN117132510A (en) * 2023-10-24 2023-11-28 临沂安保服务集团有限公司 Monitoring image enhancement method and system based on image processing

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
航天器自主交会对接的高精度目标定位方法;张莹瑾;秦世引;胡晓惠;;宇航学报(第12期);全文 *

Also Published As

Publication number Publication date
CN117372966A (en) 2024-01-09

Similar Documents

Publication Publication Date Title
CN117372966B (en) Turntable state monitoring method based on video monitoring
CN109934206A (en) A kind of rotary machinery fault diagnosis method under non-stationary operating condition
CN109902633A (en) Accident detection method and device based on the camera supervised video of fixed bit
CN101299275A (en) Method and device for detecting target as well as monitoring system
CN107742307A (en) Based on the transmission line galloping feature extraction and parameters analysis method for improving frame difference method
CN111160210A (en) Video-based water flow velocity detection method and system
CN109657600A (en) A kind of video area removes altering detecting method and device
CN115294145B (en) Method and system for measuring sag of power transmission line
CN109584232A (en) Equipment use state on-line monitoring method, system and terminal based on image recognition
CN106548158A (en) Crowd density intelligent monitor system and method based on machine vision
CN107481260A (en) A kind of region crowd is detained detection method, device and storage medium
CN113084193B (en) In-situ quality comprehensive evaluation method for selective laser melting technology
CN113884299A (en) Rotational molding machine fault detection method based on artificial intelligence
CN105447863A (en) Residue detection method based on improved VIBE
CN111597992A (en) Scene object abnormity identification method based on video monitoring
CN105426811B (en) A kind of crowd's abnormal behaviour and crowd density recognition methods
CN115063427A (en) Pollutant discharge monitoring image processing method for novel ship
CN105115987B (en) Taper roller upside-down mounting defect inspection method based on digital filtering
CN108010076A (en) A kind of end face appearance modeling method towards intensive industry bar image detection
Singh et al. HMI-guard: A platform for detecting errors in human-machine interfaces
CN117237676B (en) Method for processing small target drop track of nuclear power plant based on event camera
CN116109634B (en) Round appearance data detection system and method based on big data processing
CN116205914B (en) Waterproof coating production intelligent monitoring system
CN116958707B (en) Image classification method, device and related medium based on spherical machine monitoring equipment
CN114494979B (en) Method for video recognition of ecological flow discharge

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant