CN109561281A - Industrial equipment method for safety monitoring, device, control device and readable storage medium storing program for executing - Google Patents
Industrial equipment method for safety monitoring, device, control device and readable storage medium storing program for executing Download PDFInfo
- Publication number
- CN109561281A CN109561281A CN201811362708.6A CN201811362708A CN109561281A CN 109561281 A CN109561281 A CN 109561281A CN 201811362708 A CN201811362708 A CN 201811362708A CN 109561281 A CN109561281 A CN 109561281A
- Authority
- CN
- China
- Prior art keywords
- industrial equipment
- target
- information
- region
- multiple images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the present invention provides a kind of industrial equipment method for safety monitoring, device, control device and readable storage medium storing program for executing, belongs to security monitoring field.This method comprises: obtaining the multiple images information of multiple monitoring area ranges of multiple images collector acquisition;For each image information, the specific pixel in image information is extracted, using the specific pixel of extraction as a sub- image information;Specific pixel in the corresponding multiple subgraph information of multiple images collector corresponds to a target 3D region, the corresponding multiple monitoring area coverage goal 3D regions of multiple images collector;By multiple subgraph information inputs into default machine learning model, obtain for characterizing industrial equipment output result whether in a safe condition.The control device can obtain image information of the industrial equipment in target 3D region in real time, so as to monitor in real time to industrial equipment, to improve the safety of industrial equipment, and then avoid the waste problem of human and material resources.
Description
Technical field
The present invention relates to security monitoring field, in particular to a kind of industrial equipment method for safety monitoring, device, in
Control equipment and readable storage medium storing program for executing.
Background technique
The fields such as the engineering and industrial circle, such as electric power, chemical industry, building that have higher requirements safely at some Duis, safety
It is a long-standing demand with specification monitoring.For example, traditional does in electric power facility construction and power equipment inspection field
Method is all to arrange artificial routine inspection mode, is regularly maked an inspection tour to equipment such as high voltage towers, to confirm that equipment state is good, is not had
There are foreign body retention, invasion, destruction and periphery to construct in violation of rules and regulations.This method generally requires a large amount of human and material resources, even
The life that also will cause people threatens, to can not monitor in real time to the safety of these industrial equipments.
Summary of the invention
The embodiment of the present invention is designed to provide a kind of industrial equipment method for safety monitoring, device, control device and can
Read storage medium.
In a first aspect, the embodiment of the invention provides a kind of industrial equipment method for safety monitoring, which comprises obtain
The multiple images information of multiple monitoring area ranges of multiple images collector acquisition;For each image information, institute is extracted
The specific pixel in image information is stated, using the specific pixel of extraction as a sub- image information;Described multiple images collector
Pixel in corresponding multiple subgraph information corresponds to a target 3D region, and described multiple images collector is corresponding more
A monitoring area covers the target 3D region;By the multiple subgraph information input into default machine learning model,
It obtains for characterizing industrial equipment output result whether in a safe condition.
Optionally, it by the multiple subgraph information input into default machine learning model, obtains described for characterizing
Industrial equipment output result whether in a safe condition, comprising: by the multiple subgraph information input to default engineering
It practises in model, obtains the characteristic information of target object;Judge that the industrial equipment is according to the characteristic information of the target object
It is no in a safe condition, it obtains for characterizing industrial equipment output result whether in a safe condition.
Optionally, the default machine learning model is neural network model, convolutional neural networks model, depth nerve net
Any one in network model.
Optionally, the characteristic information of the target object includes at least following a kind of: the type of the target object, described
The quantity of target object, the status information of the target object, the target object be in the target 3D region when
Between and the target object be in the existing way in the target 3D region.
Optionally, near the industrial equipment that described multiple images collector is respectively arranged in area of space not
Same position, or the different location being set on the industrial equipment, described multiple images collector monitor the industrial equipment
Multiple monitoring area ranges at least certain area is not overlapped or partly overlaps, so that total prison of described multiple images collector
Control target 3D region described in region overlay.
Optionally, it by the multiple subgraph information input into default machine learning model, obtains described for characterizing
Industrial equipment output result whether in a safe condition, comprising: splice the multiple subgraph information, described in acquisition
Industrial equipment corresponds to the image information of the target 3D region;The industrial equipment is corresponded into the target 3D region
Image information be input in the default machine learning model, obtain for characterizing whether the industrial equipment is in safe shape
The output result of state.
Optionally, by described image information input into default machine learning model, acquisition is set for characterizing the industry
It is standby whether after output result in a safe condition, further includes: when judge that the industrial equipment is in the hole, transmission
Warning message.
Optionally, the industrial equipment is set to predetermined location, and the predetermined location is high voltage tower, power station wheel
At least one of group, petroleum pipeline, buried cable.
Second aspect, the embodiment of the invention provides a kind of industrial equipment safety monitoring device, described device includes: image
Data obtaining module, the multiple images information of multiple monitoring area ranges for obtaining the acquisition of multiple images collector;Pixel
Extraction module, for extracting the specific pixel in each image information, using the specific pixel of extraction as a subgraph
Information, wherein the specific pixel in the corresponding multiple subgraph information of described multiple images collector corresponds to a target three
Region is tieed up, the corresponding multiple monitoring areas of described multiple images collector cover the target 3D region;Analysis module,
For into default machine learning model, obtaining the multiple subgraph information input for characterizing whether industrial equipment is in
The output result of safe condition.
Optionally, the analysis module is specifically used for the multiple subgraph information input to default engineering
It practises in model, obtains the characteristic information of target object;Judge that the industrial equipment is according to the characteristic information of the target object
It is no in a safe condition, it obtains for characterizing industrial equipment output result whether in a safe condition.
Optionally, the default machine learning model is neural network model, convolutional neural networks model, depth nerve net
Any one in network model.
Optionally, the characteristic information of the target object includes at least following a kind of: the type of the target object, described
The quantity of target object, the status information of the target object, the target object be within the scope of the monitoring area when
Between and the target object be in the existing way within the scope of the monitoring area.
Optionally, near the industrial equipment that described multiple images collector is respectively arranged in area of space not
Same position, or the different location being set on the industrial equipment, described multiple images collector monitor the industrial equipment
Multiple monitoring area ranges at least certain area is not overlapped or partly overlaps, so that total prison of described multiple images collector
Control target 3D region described in region overlay.
Optionally, the analysis module, is also used to: the multiple subgraph information being spliced, described in acquisition
Industrial equipment corresponds to the image information of the target 3D region;The industrial equipment is corresponded into the target 3D region
Image information be input in the default machine learning model, obtain for characterizing whether the industrial equipment is in safe shape
The output result of state.
Optionally, described device further include: warning message sending module, for judging the industrial equipment in danger
When state, warning message is sent.
The third aspect, the embodiment of the present invention provide a kind of control device, including processor and memory, the memory
It is stored with computer-readable instruction fetch, when the computer-readable instruction fetch is executed by the processor, operation such as first party
The step in method that face provides.
Fourth aspect, the embodiment of the present invention provide a kind of readable storage medium storing program for executing, are stored thereon with computer program, feature
It is, the step in the method provided such as first aspect is provided when the computer program is executed by processor.
The beneficial effect of the embodiment of the present invention is:
The embodiment of the present invention provides a kind of industrial equipment method for safety monitoring, device, control device and readable storage medium storing program for executing,
This method obtains the multiple images information of multiple monitoring area ranges of described multiple images collector acquisition by control device,
Specific pixel is extracted to obtain multiple subgraphs;Then by the multiple subgraph information input to default machine learning model
In, obtain for characterize industrial equipment output whether in a safe condition as a result, so, control device can obtain in real time
Image information of the industrial equipment in target 3D region is set so as to monitor in real time to industrial equipment with improving industry
Standby safety, and then avoid the waste problem of human and material resources.
Other features and advantages of the present invention will be illustrated in subsequent specification, also, partly be become from specification
It is clear that by implementing understanding of the embodiment of the present invention.The objectives and other advantages of the invention can be by written theory
Specifically noted structure is achieved and obtained in bright book, claims and attached drawing.
Detailed description of the invention
In order to illustrate the technical solution of the embodiments of the present invention more clearly, below will be to needed in the embodiment attached
Figure is briefly described, it should be understood that the following drawings illustrates only certain embodiments of the present invention, therefore is not construed as pair
The restriction of range for those of ordinary skill in the art without creative efforts, can also be according to this
A little attached drawings obtain other relevant attached drawings.
Fig. 1 shows a kind of structural block diagram that can be applied to the control device in the embodiment of the present application;
Fig. 2 is a kind of flow chart of industrial equipment method for safety monitoring provided in an embodiment of the present invention;
Fig. 3 is a kind of target 3D region schematic diagram based on steel tower provided in an embodiment of the present invention;
Fig. 4 is a kind of schematic diagram being monitored using two image acquisition devices to steel tower provided in an embodiment of the present invention;
Fig. 5 is a kind of schematic diagram being monitored using four image acquisition devices to steel tower provided in an embodiment of the present invention;
Fig. 6 is a kind of signal that image mosaic is carried out using different pixel extraction modes provided in an embodiment of the present invention
Figure;
Fig. 7 is a kind of schematic diagram of spliced image information and target 3D region provided in an embodiment of the present invention;
Fig. 8 is the schematic diagram that a kind of pair of ground end cable provided in an embodiment of the present invention realizes monitoring;
Fig. 9 is a kind of structural block diagram of industrial equipment safety monitoring device provided in an embodiment of the present invention.
Specific embodiment
Below in conjunction with attached drawing in the embodiment of the present invention, technical solution in the embodiment of the present invention carries out clear, complete
Ground description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.Usually exist
The component of the embodiment of the present invention described and illustrated in attached drawing can be arranged and be designed with a variety of different configurations herein.Cause
This, is not intended to limit claimed invention to the detailed description of the embodiment of the present invention provided in the accompanying drawings below
Range, but it is merely representative of selected embodiment of the invention.Based on the embodiment of the present invention, those skilled in the art are not doing
Every other embodiment obtained under the premise of creative work out, shall fall within the protection scope of the present invention.
It should also be noted that similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi
It is defined in a attached drawing, does not then need that it is further defined and explained in subsequent attached drawing.Meanwhile of the invention
In description, term " first ", " second " etc. are only used for distinguishing description, are not understood to indicate or imply relative importance.
Fig. 1 is please referred to, Fig. 1 shows a kind of structural block diagram of control device 100 that can be applied in the embodiment of the present application.
Control device 100 may include industrial equipment safety monitoring device, memory 101, storage control 102, processor 103, outer
If interface 104, input-output unit 105, audio unit 106, display unit 107.
The memory 101, storage control 102, processor 103, Peripheral Interface 104, input-output unit 105, sound
Frequency unit 106, each element of display unit 107 are directly or indirectly electrically connected between each other, to realize the transmission or friendship of data
Mutually.It is electrically connected for example, these elements can be realized between each other by one or more communication bus or signal wire.The industry
Equipment safety monitoring device includes that at least one can be stored in the memory 101 in the form of software or firmware (firmware)
In or the software function that is solidificated in the operating system (operating system, OS) of the industrial equipment safety monitoring device
Module.The processor 103 is for executing the executable module stored in memory 101, such as industrial equipment safety prison
The software function module or computer program that control device includes.
Wherein, memory 101 may be, but not limited to, random access memory (Random Access Memory,
RAM), read-only memory (Read Only Memory, ROM), programmable read only memory (Programmable Read-Only
Memory, PROM), erasable read-only memory (Erasable Programmable Read-Only Memory, EPROM),
Electricallyerasable ROM (EEROM) (Electric Erasable Programmable Read-Only Memory, EEPROM) etc..
Wherein, memory 101 is for storing program, and the processor 103 executes described program after receiving and executing instruction, aforementioned
Method performed by the server that the stream process that any embodiment of the embodiment of the present invention discloses defines can be applied to processor 103
In, or realized by processor 103.
Processor 103 can be a kind of IC chip, the processing capacity with signal.Above-mentioned processor 103 can
To be general processor, including central processing unit (Central Processing Unit, abbreviation CPU), network processing unit
(Network Processor, abbreviation NP) etc.;Can also be digital signal processor (DSP), specific integrated circuit (ASIC),
Ready-made programmable gate array (FPGA) either other programmable logic device, discrete gate or transistor logic, discrete hard
Part component.It may be implemented or execute disclosed each method, step and the logic diagram in the embodiment of the present invention.General processor
It can be microprocessor or the processor 103 be also possible to any conventional processor etc..
Various input/output devices are couple processor 103 and memory 101 by the Peripheral Interface 104.Some
In embodiment, Peripheral Interface 104, processor 103 and storage control 102 can be realized in one single chip.Other one
In a little examples, they can be realized by independent chip respectively.
Input-output unit 105 realizes user and the server (or local terminal) for being supplied to user input data
Interaction.The input-output unit 105 may be, but not limited to, mouse and keyboard etc..
Audio unit 106 provides a user audio interface, may include one or more microphones, one or more raises
Sound device and voicefrequency circuit.
Display unit 107 provides an interactive interface (such as user's operation circle between the control device 100 and user
Face) or for display image data give user reference.In the present embodiment, the display unit 107 can be liquid crystal display
Or touch control display.It can be the capacitance type touch control screen or resistance of support single-point and multi-point touch operation if touch control display
Formula touch screen etc..Single-point and multi-point touch operation is supported to refer to that touch control display can sense on the touch control display one
Or at multiple positions simultaneously generate touch control operation, and the touch control operation that this is sensed transfer to processor 103 carry out calculate and
Processing.
Various input/output devices are couple processor 103 and memory 101 by the Peripheral Interface 104.Some
In embodiment, Peripheral Interface 104, processor 103 and storage control 102 can be realized in one single chip.Other one
In a little examples, they can be realized by independent chip respectively.
Input-output unit 105 is used to be supplied to the interaction that user input data realizes user and processing terminal.It is described defeated
Entering output unit 105 may be, but not limited to, mouse and keyboard etc..
It is appreciated that structure shown in FIG. 1 is only to illustrate, the control device 100 may also include more than shown in Fig. 1
Perhaps less component or with the configuration different from shown in Fig. 1.Each component shown in Fig. 1 can use hardware, software
Or combinations thereof realize.
Referring to figure 2., Fig. 2 is a kind of flow chart of industrial equipment method for safety monitoring provided in an embodiment of the present invention, institute
The method of stating includes the following steps:
Step S110: the multiple images information of multiple monitoring area ranges of multiple images collector acquisition is obtained.It is described
Multiple images collector corresponds to multiple monitoring areas, i.e., each image acquisition device acquires the image information of a monitoring area.
Industrial equipment method for safety monitoring in the present embodiment is applied to control device, and control device is generally positioned at and work
In the Central Control Room of industry equipment distance nearby, monitoring personnel can carry out industrial equipment by the control device in Central Control Room real
When monitor, control device in real time acquire the multiple monitoring areas of industrial equipment image information multiple images collector connect
It connects.
Multiple images collector may be disposed at the different location on industrial equipment, may also set up different around industrial equipment
Position can be used for carrying out real time image collection to industrial equipment, and multiple images collector can clap industrial equipment
According to or video recording, the image information within the scope of monitoring area so as to acquire described multiple images collector.In some implementations
In example, multiple monitoring area ranges at least certain area of multiple images collector monitoring industrial equipment is not overlapped or part weight
It is folded, so that total monitoring area of the multiple monitoring area can cover a target 3D region.In some embodiments,
The target 3D region is target monitoring region, and the shape of the target 3D region can be cuboid, cylindrical body, pros
Body, any regular three-dimensional shape or any irregular three-dimensional shape.
Industrial equipment generally refers to the medium-and-large-sized equipment of industrial purpose, it is however generally that, industrial equipment is in predetermined location
It does not have greatly changed, for example, high voltage iron tower, power station wheel group, petroleum pipeline, buried cable etc..
The multiple images corresponding to multiple monitoring areas of acquisition are sent to control device by multiple images collector, due to
Multiple images collector be to industrial equipment carry out real time image collection, so, multiple images collector send image mode
It can be and sent according to prefixed time interval, for example, multiple images collector carries out the image of acquisition or video information
Storage, is being sent to control device for image every ten minutes or one hour, certainly, multiple images collector can also be real-time
The image of acquisition is sent to control device.
Each image acquisition device is equipped with wireless module, can be wirelessly connected with control device, so that control device can be with
Obtain the image information of each image acquisition device acquisition in real time by being wirelessly transferred, in order to carry out effective monitoring to industrial equipment,
Control device can be from the image information in the selection target period in the image of acquisition.In some embodiments, the multiple
Connection type between image acquisition device or between multiple images collector and control device can be wired connection.
In addition, image acquisition device can be RGB image sensor, monocular image sensor, binocular image sensor, infrared
Imaging sensor, laser point cloud imaging sensor etc..
Step S120: for each image information, the specific pixel in described image information is extracted, by the specific of extraction
Pixel is as a sub- image information.For an image acquisition device, there may be not in the piece image of acquisition
Belong to the image information of target 3D region.Therefore, it is necessary to will not belong to the image information of target 3D region to reject, retains and belong to
In the image information of target 3D region.Particularly, the pixel in the image of acquisition is handled, rejecting is not belonging to target three
The pixel in region is tieed up, the pixel for belonging to target 3D region is retained.Treated, and pixel can be used as a subgraph
Information.
Step S130: it by the multiple subgraph information input into default machine learning model, obtains for characterizing
State industrial equipment output result whether in a safe condition.In some embodiments, the default machine learning model is mind
Through any one in network model, convolutional neural networks model, deep neural network model.
In some embodiments, it is described by the multiple subgraph information input into default machine learning model, obtain
For characterizing industrial equipment output result whether in a safe condition, comprising: by the multiple subgraph information input
Into default machine learning model, the characteristic information of target object is obtained;Institute is judged according to the characteristic information of the target object
State whether industrial equipment is in a safe condition, obtains for characterizing industrial equipment output knot whether in a safe condition
Fruit.
In some embodiments, it is described by the multiple subgraph information input into default machine learning model, obtain
For characterizing industrial equipment output result whether in a safe condition, comprising: carry out the multiple subgraph information
Splicing obtains the image information that the industrial equipment corresponds to the target 3D region;The industrial equipment is corresponded into institute
The image information for stating target 3D region is input in the default machine learning model, is obtained for characterizing the industrial equipment
Whether output result in a safe condition.In some embodiments, by the multiple subgraph information input to default machine
Before device learning model, the multiple subgraph information can be spliced, obtain the industrial equipment corresponding to the target
The image information of 3D region;And the image information that the industrial equipment corresponds to the target 3D region is input to described
In default machine learning model, obtain for characterizing industrial equipment output result whether in a safe condition.
Control device can be sentenced after the multiple subgraph information for obtaining industrial equipment based on the multiple subgraph information
Whether the industrial equipment that breaks is in a safe condition, specifically, by by the multiple subgraph information input to default engineering
It practises in model, default machine learning model can recognise that in addition to industrial equipment from the multiple subgraph information
Object close to industrial equipment, i.e., whether there is exotic invasive object to appear in the target 3D region of image acquisition device, if
There is the exotic invasive object in the image information of target time section, and the object moves closer to industrial equipment, then can determine that this
Industrial equipment is in the hole, can if any foreign body does not occur in target 3D region in the target time period
Determine that industrial equipment is in a safe condition.
Wherein, the target 3D region can with as shown in figure 3, Fig. 3 be the target 3D region schematic diagram based on steel tower,
The monitoring range of multiple images collector (not shown) is not at least partly overlapped or partly overlaps in Fig. 3, and can cover described
Target 3D region.For example, if being based on described image information, if analyzing foreign body intrusion monitoring space (i.e. target three
Tie up region) in 2, then it is assumed that the foreign body produces certain threat to steel tower, i.e. judgement steel tower is in the hole, monitoring
Space can be different according to the type or monitoring needs of threat, and space 1 is monitored in Fig. 3 is greater than monitoring space 2, and two monitorings are empty
The interior monitoring requirements to space are different, for example, then monitoring space can be smaller, and personnel or construction if it is foreign body intrusion
It is live then need to be monitored in the larger context.
Wherein, characterizing industrial equipment output result whether in a safe condition can carry out table in any manner
Show, such as output result is that 1 expression industrial equipment is in a safe condition, output result indicates that industrial equipment is in non-security for 0
State, or output result can also be the prompt information of " safety "/" dangerous ".
So in the present embodiment, the multiple images information of the monitoring area by obtaining the acquisition of multiple images collector, warp
It crosses processing and obtains the subgraph information of target three-dimensional space (monitoring area), then by the multiple subgraph information input in advance
If in machine learning model, obtaining for characterizing industrial equipment output whether in a safe condition as a result, further,
Control device can obtain industrial equipment corresponding to the spliced image information in target 3D region, so as to industry in real time
Device target three-dimensional space is monitored in real time, to improve the safety of industrial equipment.
In order to carry out conduct monitoring at all levels to industrial equipment, described image collector is multiple, multiple described image collectors
It is respectively arranged at the different location of area of space near the industrial equipment, multiple described image collectors monitor the industry
The monitoring area of equipment is not at least partly overlapped or partly overlaps, so that total monitoring area of multiple described image collectors is
The monitoring area range.
Multiple images collector can be placed in the relevant different location of industrial equipment, make it possible to identify industrial equipment phase
A region in space is closed, at least certain area is not overlapped the image that multiple images collector obtains in monitoring area, this
When only need to set the quantity and distributing position of image acquisition device, it will be able to so that the monitoring area group of its multiple images collector
Synthesize total monitoring area range.
Industrial equipment (such as steel tower) is monitored for two image acquisition devices referring to figure 4. and in Fig. 5, Fig. 4, two figures
Picture collector is not exclusively overlapped in monitoring area towards steel tower lower section, at this point, the monitored space that two image acquisition devices are total
Domain covers the target 3D region (cylindrical region), is that four image acquisition devices are placed on a square around in Fig. 5
On four sides of shape, two image acquisition devices in left and right are symmetrical arranged, bottom image acquisition device upward, top image collector
Downward, at this point, the total monitoring area of four image acquisition devices can cover target 3D region (the rectangular space area
Domain).
As an implementation, in order to be identified to subtle target object, point of each image acquisition device
Resolution is greater than default resolution value.
As an implementation, clearly scheme to can also obtain industrial equipment in the case where the bad weather such as rainy day
As information, then each image acquisition device is mounted in transparency protected device, also, transparency protected device can not only make image
Collector can also acquire clearly image under the bad weathers such as rainy day, and can also extend the use of image acquisition device
Service life allows image acquisition device not by the influence of external environment.Wherein, transparency protected device can be for outside transparent plastic
Shell etc..
In addition, as an implementation, image acquisition device may include master image collector and standby image acquisition device, institute
It states master image collector and the standby image acquisition device to be all wirelessly connected with the control device, when control device detects master
When image acquisition device is in abnormal condition, controls the standby image acquisition device and open, to be adopted by the standby image acquisition device
Collect the image information of the industrial equipment in the target time period.
Wherein, the setting position of standby image acquisition device is normally at the side of master image collector, so that two images
The monitoring area of collector can be largely overlapped, if image acquisition device is multiple, each image acquisition device includes master
Image acquisition device and standby image acquisition device.
Wherein, the abnormal condition of master image collector refers to that master image collector is in malfunction and can not carry out image
It acquires or master image collector can not be communicated with control device or the electricity of master image collector is used up etc. shape
State then enables standby image acquisition device at this time and continues to adopt so that control device can not obtain the image of master image collector acquisition
The image information for collecting industrial equipment, then can make the acquisition of image information not interrupt, so that control device can set industry
It is standby continuously and uninterruptedly to be monitored.
During control device obtains the image information of industrial equipment in the target time period as a result, control device is first
The multiple images information of the industrial equipment of first available multiple described image collector acquisitions, then from the multiple figure
As extracting pixel in information, multiple subgraph information are determined;Then the multiple subgraph information is spliced, described in acquisition
Industrial equipment corresponds to the image information of the target 3D region.
Wherein, process image spliced are as follows: each image is pre-processed first, it is described pretreatment include from
Certain frame is extracted in the multiple images frame of timing, and multiple corresponding pixel groups are extracted from the picture frame of extraction, it is described
Each pixel group is to belong to the pixel set of target three-dimensional space, and one pixel group can be used as a sub- image information,
Then the multiple subgraph is spliced, the image zooming-out side for extracting pixel group and being obtained for each image acquisition device
Formula may be different, for example, a simple extracting method are as follows: according to a coordinate and size, a company is extracted from picture frame
Continuous two-dimensional array, Fig. 6 give the pixel extraction of different images collector (i.e. sensor in Fig. 6), specifically
Frame and pixel extraction system are provided by control device, control device according to the position and monitoring area of each image acquisition device come
The extracting mode of each image acquisition device is calculated, and further multiple subgraph information of extraction are spliced into described in a correspondence
The image information of target 3D region although specific calculating process is complex, and may need manual calibration in advance,
It is since the specification of industrial equipment (such as steel tower) is often fixed, primary calculate can be so that extracting method be suitable for largely
The monitoring of industrial equipment.The image information for finally splicing acquisition corresponds to entire target 3D region, that is to say, that appears in this
Target object in a target 3D region will be presented on a corresponding position in spliced image information, and target is three-dimensional
The boundary in region is corresponding with a boundary in spliced image information.
Fig. 7 gives the schematic diagram of above-mentioned spliced image information and target 3D region, it should be noted that splicing
An industrial equipment image corresponding with real industrial device shaped must can be not necessarily obtained in image information afterwards, it is also possible to
Any splicing of the different components of industrial equipment.
In the above-described embodiments, the default machine learning model is neural network model, convolutional neural networks model, depth
Spend any one in neural network model.Control device can be by the multiple subgraph information input to default machine learning
In model, obtain for characterizing industrial equipment output whether in a safe condition as a result, wherein it is possible to by presetting machine
Device learning model identifies in image information whether there is target object, if there is target object, in order to further judge the target
Whether object jeopardizes the safety of industrial equipment, then target object can be identified from image information by default machine learning model
Characteristic information, then judge whether the industrial equipment in a safe condition, obtains according to the characteristic information of the target object
Industrial equipment output result whether in a safe condition must be used to characterize, wherein the characteristic information packet of the target object
Include at least following a kind of: the type of the target object, the quantity of the target object, the status information of the target object,
The target object is in the time in the target 3D region and the target object is in the target 3D region
Existing way.
Wherein, the type of target object can be such as steel tower, people, vehicle, plant, animal, plastics, metal object, mesh
The status information for marking object can be that such as vehicle drive into, be constructed, personnel climb;The target object is in monitored space
Existing way within the scope of domain can be such as sleep mode, motion mode.
In order to be identified using default machine learning model to image information, default machine learning model is carried out in advance
Training, above-mentioned identical image collector setting and image contract can be used in training data and joining method obtains training data,
And the label target object in training data, label target object include the type of target object, such as steel tower, people, vehicle, plant
Object, animal, plastics, metal object etc., can also be with the state of label target object, such as vehicle drives into, constructed, Ren Yuanpan
Climb etc..Based on the above training data, default machine learning model can identify corresponding mesh in the subgraph information of input
Mark the characteristic information of object and target object.
Wherein, after obtaining training data, the model based on training method one blank of training or pre-training, this was trained
Journey can be based on such as BP algorithm or the TensorFlow frame system provided using Google, the knowledge that system is exported according to model
Other result (i.e. the characteristic information of target object).
Then control device may determine that whether the industrial equipment is in a safe condition, if such as the target object
Characteristic information is that a people appears in the target 3D region of image acquisition device, then also can determine whether that the people is in target space
Time in domain, whether more than one preset time then judged that industrial equipment is in the hole when to be, output is used for table
Levy the output result that the industrial equipment is in non-secure states (i.e. precarious position).When industrial equipment is in the hole,
The also controllable alarm device of control device sounds an alarm.If the personnel only occur in several frame images, work can determine that
Industry equipment is in a safe condition, then does not sound an alarm.
In one embodiment, it needs to judge whether industrial equipment is located according to the assembled state information of different target object
In safe condition, for example, not issuing police then when control device identification target object is that kite occurs in target 3D region
Report, but if kite carry sounds an alarm on steel tower.
It in one embodiment, can be by monitoring multiple and different target 3D regions, for example, biggish at one
When identification is without related personnel in monitoring area, does not then sound an alarm, but identify construction work vehicle in the monitoring area
When, then it sounds an alarm.
So industrial equipment safe condition judgement can according to the type of target object, state, quantity, there are the time,
Existing way etc. (i.e. the characteristic information of target object) comprehensive descision.
In addition, ground end cable is generally placed in ground end as shown in figure 8, Fig. 8 is the schematic diagram for realizing monitoring to ground end cable
Within tunnel, due to the typically no lighting system in ground end tunnel, Image Acquisition can be carried out using infrared sensor, may be used also
By multiple images collector and corresponding image contract and mode is integrated and (splices), control device can obtain more electricity
Whole outside surface image information of cable, whether being then based on image information, to judge the cable in a safe condition, for example, working as cable
When surface occurs damaged, the breakage can be identified by infrared image, when cable inner heat exception, the infrared image of cable
Color intensity can also change, at this time control device can be identified according to image information cable cross heat anomaly, and
Monitoring area is closed by setting, monitoring area is concentrated on into the relevant enclosure space of cable, therefore will not be due to other external worlds
Heat change and judge by accident cable overheat.Accurately judgement can be realized to the safe condition of cable as a result,.
In addition, as another embodiment, when judging that the industrial equipment is in the hole, control device may be used also
To send warning message, the monitoring personnel of Central Control Room is allowed to know that industrial equipment is in the hole by warning message.
Alternatively, warning message can be also sent to the use with the industrial equipment apart from nearest monitoring personnel by control device
In the terminal of family.Specifically, control device searches for monitor around industrial equipment when judging that industrial equipment is in the hole
Member obtains the range information of monitoring personnel and industrial equipment around industrial equipment, the monitor for then selecting range information nearest
Member then sends warning message to the user terminal of the monitoring personnel, and monitoring personnel can exclude the danger of industrial equipment in time as a result,
State, and then guarantee the safety of industrial equipment.
Alternatively, if after warning message is sent to the user terminal apart from nearest monitoring personnel by control device, if one
It fixes time in section, which is in precarious position, i.e. the monitoring personnel dangerous shape that excludes industrial equipment not in time
State, then warning message can be sent to the user terminal of the monitoring personnel close with industrial equipment distance second by control device again,
So that the monitoring personnel carries out precarious position exclusion to industrial equipment.
It can guarantee that in time industrial equipment is converted to safe condition from precarious position as a result, it is ensured that the operation of industrial equipment
Safety.
Fig. 9 is please referred to, Fig. 9 is a kind of structural frames of industrial equipment safety monitoring device 200 provided in an embodiment of the present invention
Figure, the device run on above-mentioned control device, and described device includes:
Image information acquisition module 210, for obtain multiple images collector acquisition multiple monitoring area ranges it is more
A image information;
Pixel extraction module 220, for extracting the specific pixel in each image information, by the specific pixel of extraction
As a sub- image information, wherein the specific pixel pair in the corresponding multiple subgraph information of described multiple images collector
Ying Yuyi target 3D region, the corresponding multiple monitoring areas of described multiple images collector cover the target space
Domain;
Analysis module 230, for into default machine learning model, obtaining the multiple subgraph information input
For characterizing industrial equipment output result whether in a safe condition.
Optionally, the analysis module 220 is specifically used for the multiple subgraph information input to default machine
In learning model, the characteristic information of the target object is obtained;The industry is judged according to the characteristic information of the target object
Whether equipment is in a safe condition, obtains for characterizing industrial equipment output result whether in a safe condition.
Optionally, the default machine learning model is neural network model, convolutional neural networks model, depth nerve net
Any one in network model.
Optionally, the characteristic information of the target object includes at least following a kind of: the type of the target object, described
The quantity of target object, the status information of the target object, the target object be in the target 3D region when
Between and the target object be in the existing way in the target 3D region.
Optionally, near the industrial equipment that described multiple images collector is respectively arranged in area of space not
Same position, or the different location being set on the industrial equipment, described multiple images collector monitor the industrial equipment
Multiple monitoring areas are not overlapped or partly overlap, so that total monitoring area of described multiple images collector covers the target
3D region.
Optionally, the analysis module 230, is specifically used for:
The multiple subgraph information is spliced, obtains the industrial equipment corresponding to the target 3D region
Image information;
The image information that the industrial equipment corresponds to the target 3D region is input to the default machine learning
In model, obtain for characterizing industrial equipment output result whether in a safe condition.
Optionally, described device further include:
Warning message sending module, for sending warning message when judging that the industrial equipment is in the hole.
Optionally, the resolution ratio of each image acquisition device is greater than default resolution value in described image collector.
Optionally, each image acquisition device is mounted in transparency protected device.
Optionally, described image collector includes master image collector and standby image acquisition device, the master image acquisition
Device and the standby image acquisition device are all wirelessly connected with the control device, described device further include:
Standby image acquisition device opening module, for controlling when detecting that the master image collector is in abnormal condition
It makes the standby image acquisition device to open, to acquire the industrial equipment in the target time period by the standby image acquisition device
Image information.
The embodiment of the present invention provides a kind of read/write memory medium, when the computer program is executed by processor, executes
Method process performed by control device in embodiment of the method as shown in Figure 2.
It is apparent to those skilled in the art that for convenience and simplicity of description, the device of foregoing description
Specific work process, no longer can excessively be repeated herein with reference to the corresponding process in preceding method.
In conclusion the embodiment of the present invention provides a kind of industrial equipment method for safety monitoring, device, control device and readable
Storage medium, this method by control device obtain described multiple images collector acquire multiple monitoring area ranges it is multiple
Image information extracts specific pixel to obtain multiple subgraphs;Then by the multiple subgraph information input to default machine
In learning model, obtain for characterize industrial equipment output whether in a safe condition as a result, so, control device can
Image information of the industrial equipment in target 3D region is obtained in real time, so as to monitor in real time to industrial equipment, to mention
The safety of high industrial equipment, and then avoid the waste problem of human and material resources.
In several embodiments provided herein, it should be understood that disclosed device and method can also pass through
Other modes are realized.The apparatus embodiments described above are merely exemplary, for example, flow chart and block diagram in attached drawing
Show the device of multiple embodiments according to the present invention, the architectural framework in the cards of method and computer program product,
Function and operation.In this regard, each box in flowchart or block diagram can represent the one of a module, section or code
Part, a part of the module, section or code, which includes that one or more is for implementing the specified logical function, to be held
Row instruction.It should also be noted that function marked in the box can also be to be different from some implementations as replacement
The sequence marked in attached drawing occurs.For example, two continuous boxes can actually be basically executed in parallel, they are sometimes
It can execute in the opposite order, this depends on the function involved.It is also noted that every in block diagram and or flow chart
The combination of box in a box and block diagram and or flow chart can use the dedicated base for executing defined function or movement
It realizes, or can realize using a combination of dedicated hardware and computer instructions in the system of hardware.
In addition, each functional module in each embodiment of the present invention can integrate one independent portion of formation together
Point, it is also possible to modules individualism, an independent part can also be integrated to form with two or more modules.
It, can be with if the function is realized and when sold or used as an independent product in the form of software function module
It is stored in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantially in other words
The part of the part that contributes to existing technology or the technical solution can be embodied in the form of software products, the meter
Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be a
People's computer, server or network equipment etc.) it performs all or part of the steps of the method described in the various embodiments of the present invention.
And storage medium above-mentioned includes: that USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), arbitrary access are deposited
The various media that can store program code such as reservoir (RAM, Random Access Memory), magnetic or disk.
The foregoing is only a preferred embodiment of the present invention, is not intended to restrict the invention, for the skill of this field
For art personnel, the invention may be variously modified and varied.All within the spirits and principles of the present invention, made any to repair
Change, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.It should also be noted that similar label and letter exist
Similar terms are indicated in following attached drawing, therefore, once being defined in a certain Xiang Yi attached drawing, are then not required in subsequent attached drawing
It is further defined and explained.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can easily think of the change or the replacement, and should all contain
Lid is within protection scope of the present invention.Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to
Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those
Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment
Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that
There is also other identical elements in process, method, article or equipment including the element.
Claims (11)
1. a kind of industrial equipment method for safety monitoring, which is characterized in that the described method includes:
Obtain the multiple images information of multiple monitoring areas of multiple images collector acquisition;
For each image information, the specific pixel in described image information is extracted, using the specific pixel of extraction as one
Subgraph information;
Specific pixel in the corresponding multiple subgraph information of described multiple images collector corresponds to a target 3D region,
The corresponding multiple monitoring areas of described multiple images collector cover the target 3D region;
By the multiple subgraph information input into default machine learning model, obtain for characterizing whether industrial equipment is in
The output result of safe condition.
2. the method according to claim 1, wherein by the multiple subgraph information input to default engineering
It practises in model, obtains for characterizing industrial equipment output result whether in a safe condition, comprising:
By the multiple subgraph information input into default machine learning model, the characteristic information of target object is obtained;
Judge whether the industrial equipment is in a safe condition according to the characteristic information of the target object, obtains for characterizing
State industrial equipment output result whether in a safe condition.
3. according to the method described in claim 2, it is characterized in that, the default machine learning model be neural network model,
Any one in convolutional neural networks model, deep neural network model.
4. according to the method described in claim 2, it is characterized in that, the characteristic information of the target object includes at least with next
Kind: the type of the target object, the quantity of the target object, the status information of the target object, the target object
Time and the target object in the target 3D region are in the existing way in the target 3D region.
5. the method according to claim 1, wherein the work that described multiple images collector is respectively arranged at
Different location near industry equipment in area of space, or the different location being set on the industrial equipment, the multiple figure
As multiple monitoring area ranges at least certain area that collector monitors the industrial equipment is not overlapped or partly overlaps, so that
Total monitoring area of described multiple images collector covers the target 3D region.
6. the method according to claim 1, wherein by the multiple subgraph information input to default engineering
It practises in model, obtains for characterizing industrial equipment output result whether in a safe condition, comprising:
The multiple subgraph information is spliced, the image that the industrial equipment corresponds to the target 3D region is obtained
Information;
The image information that the industrial equipment corresponds to the target 3D region is input to the default machine learning model
In, it obtains for characterizing industrial equipment output result whether in a safe condition.
7. -6 any method according to claim 1, which is characterized in that by the multiple subgraph information input to presetting
In machine learning model, obtain for after characterizing industrial equipment output result whether in a safe condition, further includes:
When judging that the industrial equipment is in the hole, warning message is sent.
8. -6 any method according to claim 1, which is characterized in that the industrial equipment is set to predetermined location, institute
Stating predetermined location is at least one of high voltage tower, power station wheel group, petroleum pipeline, buried cable.
9. a kind of industrial equipment safety monitoring device, which is characterized in that described device includes:
Image information acquisition module, the multiple images letter of multiple monitoring area ranges for obtaining the acquisition of multiple images collector
Breath;
Pixel extraction module, for extracting the specific pixel in each image information, using the specific pixel of extraction as one
A sub- image information, wherein the specific pixel in the corresponding multiple subgraph information of described multiple images collector corresponds to one
A target 3D region, the corresponding multiple monitoring areas of described multiple images collector cover the target 3D region;
Analysis module, for into default machine learning model, obtaining and being used for table the multiple subgraph information input
Levy industrial equipment output result whether in a safe condition.
10. a kind of control device, which is characterized in that including processor and memory, the memory is stored with computer can
Instruction is read, when the computer-readable instruction fetch is executed by the processor, operation side as described in claim 1-8 is any
Step in method.
11. a kind of readable storage medium storing program for executing, is stored thereon with computer program, which is characterized in that the computer program is processed
Operation such as the step in any the method for claim 1-8 when device executes.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811362708.6A CN109561281A (en) | 2018-11-15 | 2018-11-15 | Industrial equipment method for safety monitoring, device, control device and readable storage medium storing program for executing |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811362708.6A CN109561281A (en) | 2018-11-15 | 2018-11-15 | Industrial equipment method for safety monitoring, device, control device and readable storage medium storing program for executing |
Publications (1)
Publication Number | Publication Date |
---|---|
CN109561281A true CN109561281A (en) | 2019-04-02 |
Family
ID=65866617
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201811362708.6A Pending CN109561281A (en) | 2018-11-15 | 2018-11-15 | Industrial equipment method for safety monitoring, device, control device and readable storage medium storing program for executing |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN109561281A (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110927731A (en) * | 2019-11-15 | 2020-03-27 | 深圳市镭神智能系统有限公司 | Three-dimensional protection method, three-dimensional detection device and computer readable storage medium |
CN112748734A (en) * | 2020-12-18 | 2021-05-04 | 河北润忆安全技术服务有限公司 | Self-adaptive inspection method and inspection system |
CN113112427A (en) * | 2021-04-16 | 2021-07-13 | 广东韶钢松山股份有限公司 | Method, device and equipment for monitoring molten iron transportation and storage medium |
CN115879897A (en) * | 2023-03-03 | 2023-03-31 | 中国水利水电科学研究院 | Water conservancy and hydropower engineering construction management system and management method |
CN116909196A (en) * | 2023-08-31 | 2023-10-20 | 国网山东省电力公司济南供电公司 | Intelligent cable channel monitoring system and method based on Internet of things technology |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106530569A (en) * | 2016-10-17 | 2017-03-22 | 北京小米移动软件有限公司 | Method and device for fire monitoring |
CN106973255A (en) * | 2016-01-14 | 2017-07-21 | 维布络有限公司 | Region-of-interest monitoring method and its supervising device |
US20170318283A1 (en) * | 2016-04-27 | 2017-11-02 | Disney Enterprises, Inc. | Systems and Methods for Creating an Immersive Video Content Environment |
CN107729807A (en) * | 2017-09-05 | 2018-02-23 | 南京理工大学 | Integrated external force damage prevention target identification and intelligent early-warning system |
CN108765870A (en) * | 2018-06-28 | 2018-11-06 | 信利光电股份有限公司 | Interior method for safety monitoring, device, equipment and computer readable storage medium |
-
2018
- 2018-11-15 CN CN201811362708.6A patent/CN109561281A/en active Pending
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN106973255A (en) * | 2016-01-14 | 2017-07-21 | 维布络有限公司 | Region-of-interest monitoring method and its supervising device |
US20170318283A1 (en) * | 2016-04-27 | 2017-11-02 | Disney Enterprises, Inc. | Systems and Methods for Creating an Immersive Video Content Environment |
CN106530569A (en) * | 2016-10-17 | 2017-03-22 | 北京小米移动软件有限公司 | Method and device for fire monitoring |
CN107729807A (en) * | 2017-09-05 | 2018-02-23 | 南京理工大学 | Integrated external force damage prevention target identification and intelligent early-warning system |
CN108765870A (en) * | 2018-06-28 | 2018-11-06 | 信利光电股份有限公司 | Interior method for safety monitoring, device, equipment and computer readable storage medium |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110927731A (en) * | 2019-11-15 | 2020-03-27 | 深圳市镭神智能系统有限公司 | Three-dimensional protection method, three-dimensional detection device and computer readable storage medium |
CN110927731B (en) * | 2019-11-15 | 2021-12-17 | 深圳市镭神智能系统有限公司 | Three-dimensional protection method, three-dimensional detection device and computer readable storage medium |
CN112748734A (en) * | 2020-12-18 | 2021-05-04 | 河北润忆安全技术服务有限公司 | Self-adaptive inspection method and inspection system |
CN113112427A (en) * | 2021-04-16 | 2021-07-13 | 广东韶钢松山股份有限公司 | Method, device and equipment for monitoring molten iron transportation and storage medium |
CN113112427B (en) * | 2021-04-16 | 2024-05-10 | 广东韶钢松山股份有限公司 | Monitoring method, device, equipment and storage medium for molten iron transportation |
CN115879897A (en) * | 2023-03-03 | 2023-03-31 | 中国水利水电科学研究院 | Water conservancy and hydropower engineering construction management system and management method |
CN116909196A (en) * | 2023-08-31 | 2023-10-20 | 国网山东省电力公司济南供电公司 | Intelligent cable channel monitoring system and method based on Internet of things technology |
CN116909196B (en) * | 2023-08-31 | 2024-03-15 | 国网山东省电力公司济南供电公司 | Intelligent cable channel monitoring system and method based on Internet of things technology |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109561281A (en) | Industrial equipment method for safety monitoring, device, control device and readable storage medium storing program for executing | |
CN101577812B (en) | Method and system for post monitoring | |
US20230005176A1 (en) | Throwing position acquisition method and apparatus, computer device and storage medium | |
CN106408833A (en) | Perimeter intrusion detection method and system | |
CN108702485A (en) | Privacy is protected in video monitoring system | |
CN108872781A (en) | Analysis method and device based on electric power facility intelligent patrol detection | |
CN109672863A (en) | A kind of construction personnel's safety equipment intelligent monitoring method based on image recognition | |
CN112990057A (en) | Human body posture recognition method and device and electronic equipment | |
CN104796667A (en) | Campus night-vision three-dimensional virtual monitoring system and method | |
CN102982639A (en) | Video monitoring optical fiber fence system and using method thereof | |
CN105530456A (en) | Method, device and system for substation monitoring | |
CN110428400A (en) | A kind of localization method of equipment fault, device, equipment and storage medium | |
CN112541393A (en) | Transformer substation personnel detection method and device based on deep learning | |
CN113469654B (en) | Multi-level safety control system of transformer substation based on intelligent algorithm fuses | |
CN115393566A (en) | Fault identification and early warning method and device for power equipment, storage medium and equipment | |
CN115457446A (en) | Abnormal behavior supervision system based on video analysis | |
CN112688434A (en) | Monitoring and early warning method and device for power transmission and distribution line, computer equipment and medium | |
CN106710651B (en) | A kind of method and system for showing nuclear power station center accident condition transition | |
CN106650594A (en) | Video fire detection method, device and system | |
CN116846059A (en) | Edge detection system for power grid inspection and monitoring | |
Chang et al. | Safety risk assessment of electric power operation site based on variable precision rough set | |
CN104574729A (en) | Alarming method, device and system | |
CN117351462A (en) | Construction operation detection model training method, device, equipment and storage medium | |
CN202916936U (en) | Video monitoring optical fiber fence system | |
CN116580514A (en) | Intelligent security method, system, medium and electronic equipment based on Internet of things |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
RJ01 | Rejection of invention patent application after publication | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190402 |