CN108471497A - A kind of ship target real-time detection method based on monopod video camera - Google Patents

A kind of ship target real-time detection method based on monopod video camera Download PDF

Info

Publication number
CN108471497A
CN108471497A CN201810173195.8A CN201810173195A CN108471497A CN 108471497 A CN108471497 A CN 108471497A CN 201810173195 A CN201810173195 A CN 201810173195A CN 108471497 A CN108471497 A CN 108471497A
Authority
CN
China
Prior art keywords
ship
video camera
monopod video
detection
trained
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201810173195.8A
Other languages
Chinese (zh)
Inventor
刘书鹏
王闪闪
刘建宏
金倩宜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin Yaan Technology Co Ltd
Original Assignee
Tianjin Yaan Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin Yaan Technology Co Ltd filed Critical Tianjin Yaan Technology Co Ltd
Priority to CN201810173195.8A priority Critical patent/CN108471497A/en
Publication of CN108471497A publication Critical patent/CN108471497A/en
Pending legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60Control of cameras or camera modules
    • H04N23/64Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/20Analysis of motion
    • G06T7/246Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/18Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30232Surveillance

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The present invention relates to a kind of ship target real-time detection method based on monopod video camera, including video flowing of the intake comprising ship target, training data pretreatment, the calculating speed for determining training pattern and improving trained ship detection model, it is trained according to actual conditions model adjustment, by trained ship detection model and monopod video camera linkage step.This method is by video flowing being trained data prediction, determining training pattern and improving the calculating speed of trained ship detection model, adjusting training model, finally trained ship detection model and monopod video camera are linked, finally the ship object width being presented in display main bit stream is kept to account for one third of picture width or so, and be shown centered on, realize the purpose that monopod video camera persistently tracks.Design is scientific and reasonable for the method for the present invention, is skillfully constructed, is easily achieved, and effectively increases accuracy rate, and arithmetic speed is fast, the value with wide popularization and application.

Description

A kind of ship target real-time detection method based on monopod video camera
Technical field
The invention belongs to technical field of video monitoring, especially a kind of ship target based on monopod video camera side of detection in real time Method.
Background technology
Development based on deep learning in recent years, in industrial embedded product field, due to the computing capability of computing element Enhancing, product obtain the development advanced by leaps and bounds in image recognition and two big algorithm direction of speech recognition.
Artificial intelligence application is generated on product intellectual product task be desirable to product can be to biographies such as video cameras The data that sensor obtains understand and the rational thinking possessed by the mankind in a manner of make decision, to help the mankind to complete The decision-making work of various uninteresting repetitions.The existing equipment using traditional images processing method mostly only moves up remote sea Moving-target identifies, and there is the difficulty of mobile target classification, especially camera lens further after complex background under big object ship detection deposit In difficulty.
And existing training pattern is effective to big target detection but to distal end Small object, especially haze weather in recent years Ship shape object cannot detect.The high deep learning method training pattern of detection discrimination leads to single frames due to computationally intensive mostly Processing speed is slow, and live video stream detection is carried out in front end embedded platform it is difficult to apply.Cannot be real-time, it is unable to zoom identification, Cannot apply headend equipment ensure user data privacy the shortcomings of, be all unsuitable for Sea Surface Ship observation identification alarm and Track demand.
Invention content
It is an object of the invention to overcome the deficiencies of the prior art and provide a kind of, and the ship target based on monopod video camera is real-time Detection method, in the guarantee camera lens object no matter radar can be provided whether haze weather orientation object ship object it is real-time Detecting and tracking method.
The scheme of the invention is be achieved:
A kind of ship target real-time detection method based on monopod video camera, includes the following steps:
Step 1: including the video flowing of ship target by monopod video camera intake
The video stream packets of intake contain main bit stream and auxiliary code stream:
(1) main bit stream of rtsp video flowings for clearly indicating real-time video in the display;
(2) the auxiliary code stream of rtsp video flowings is handled as the input video stream of ship detection, and carries out target inspection It surveys;
Step 2: training data pre-processes
The source of training data picture includes two parts:
A part of image credit is extracted in by the picture for containing ship target from main bit stream, that is, after obtaining rtsp video flowings Resolution processes are compressed and changed to picture using the method batch of image procossing, ensure that key message is not lost;
Another part picture then comes from the ship target in extraction common data sets such as Pascal VOC, arranges needs Part, this part are mainly used for improving the generalization of ship detection, the recognition detection for ship target after furthering to zoom;
The above-mentioned picture classification handled well is arranged, ensure distal end ship type Small object in training set, various types of STOWAGE PLAN piece and Possible error detection picture accounts for the certain ratio of training data;
Step 3: determining training pattern and improving the calculating speed of trained ship detection model
The darknet frame structures in linux system are applied in selection, facilitate the later stage using c/c++ language embedded It transplanting in equipment and is docked with monopod video camera, using the principle of yolo trains ship detection model;
Image is RGB triple channel images in auxiliary code stream in the step 1;
The method for improving detection speed is, using one type objects of training ship target, and using the volume base core of multiple 1*1 sizes The principle of the convolution kernel of a n*n is substituted, meanwhile, auxiliary code stream input image size is changed by image processing algorithm, and ensure Key message is not lost, and calculative pixel number is reduced, and the method by reducing calculation amount improves calculating speed;
Step 4: being trained model adjustment according to actual conditions
Training data strictly screens, and meets error detection factor that specific environment ship detection and this environment may introduce not It introduces;
Step 5: trained ship detection model and monopod video camera are linked
By trained ship detection model transplantations to monopod video camera, the problem of in order to improve accuracy, statistics is simultaneously It filters out most believable coordinate and monopod video camera is issued in size preparation.
Moreover, in the step 5, the method for screening most credible coordinate and size is:
Present frame records coordinate (continuous record 20-25 frame of the ship target in entire picture at this time if there are object ship Target's center's point coordinates), often get 20 groups of data, data processings just carried out to this 20 groups of data, will most believable coordinate Point information is sent to monopod video camera by serial ports, and monopod video camera uses 3D algorithms, believes according to the most credible coordinate received The rotation of the length and width dimensions information of breath and ship target and zoom, monopod video camera, which remains, at this time is presented in display main bit stream Ship object width account for one third of picture width or so, and keep placed in the middle, and lasting tracking.
The advantages and positive effects of the present invention are:
1, this method is by being trained data prediction to video flowing, determining training pattern and improving the inspection of training ship target The calculating speed of model, adjusting training model are surveyed, finally by trained ship detection model and monopod video camera Linkage finally keeps the ship object width being presented in display main bit stream to account for one third of picture width or so, and placed in the middle It has been shown that, realizes the purpose that monopod video camera persistently tracks.
2, the method for the present invention improves working efficiency and the work comfort sense of staff, make staff can with having time and Energy does more and its relevant excelsior action of profession, for example personnel in charge of the case faster can more timely obtain Clue helps each post staff preferably to realize that the industry with craftsman's spirit is promoted.
3, design is scientific and reasonable, is skillfully constructed, is easily achieved for the method for the present invention, effectively increases accuracy rate, and operation Speed is fast, the value with wide popularization and application.
Description of the drawings
Fig. 1 is the flow chart of the present invention;
Fig. 2 is the specific workflow figure of the present invention;
Fig. 3 is the calculating speed block diagram that training ship detection model is improved in Fig. 1;
Fig. 4 is the design sketch of the method for the present invention intake image.
Specific implementation mode
The invention will be further described below in conjunction with the accompanying drawings and by specific embodiment.
A kind of ship target real-time detection method based on monopod video camera, as shown in Figure 1, including the following steps:
Step 1: including the video flowing of ship target by monopod video camera intake
Monopod video camera shooting positioned at ground is likely to occur the video of ship type object at a distance, and can after the zoom that furthers The video of spot target can be gone out, at this time the usually shared pixel ratio very little of ship target in the camera lens of monopod video camera, sometimes Due to haze weather, the not high problem of color contrast is also will present out, can only about show the picture effect of shape.(referring to Fig. 4, left side are the auxiliary code stream video sectional drawings of monopod video camera intake, and right side is auxiliary code stream detection identification effect of this method to intake Fruit shows)
The video stream packets of intake contain main bit stream and auxiliary code stream:
1, the main bit stream of rtsp video flowings for clearly indicating real-time video in the display;
2, the auxiliary code stream of rtsp video flowings is handled as the input video stream of ship detection, and carries out target inspection It surveys;
Step 2: training data pre-processes
The source of training data picture includes two parts:
A part of image credit is extracted in by the picture for containing ship target from main bit stream, that is, after obtaining rtsp video flowings The processing such as resolution ratio are compressed and changed to picture using the method batch of image procossing, ensure that key message is not lost;
Another part picture then comes from the ship target in extraction common data sets such as Pascal VOC, arranges needs Part, this part are mainly used for improving the generalization of ship detection, the recognition detection for ship target after furthering to zoom.
The above-mentioned picture classification handled well is arranged, ensures distal end ship type Small object, variety classes STOWAGE PLAN piece in training set The certain ratio of training data is accounted for possible error detection picture (bird of such as water surface, beacon, the building etc. on opposite bank).
Step 3: determining training pattern and improving the calculating speed of trained ship detection model
The darknet frame structures in linux system are applied in selection, it is therefore an objective to the later stage be facilitated to exist using c/c++ language It transplanting on embedded device and is docked with monopod video camera, using the principle of yolo trains ship detection model.
Image is RGB triple channel images in auxiliary code stream in the step 1, if wishing just to need video bag each second in real time Containing images more than 20 frames, and being usually detected processing is carried out on the headend equipment of low operational capability, therefore for ship The single frames processing speed of target detection will be as quickly as possible.
To improve detection speed (as shown in Figure 3), using one type objects of training ship target, network structure is relatively easy, and Calculation amount is reduced using the principle of the convolution kernel of one n*n of volume base nuclear subsitution of multiple 1*1 sizes, meanwhile, pass through image procossing Algorithm changes auxiliary code stream input image size, and ensures that key message is not lost, and calculative pixel number is reduced, by subtracting The method of few calculation amount, improves calculating speed.
Step 4: being trained model adjustment according to actual conditions
Since trained network is relatively easy, adds new feature entrance and be very easy to, but subtracting for error detection feature It is not easy to less, it is therefore desirable to which the stringent screening for noticing training data will meet specific environment ship detection and this environment can The error detection factor that can be introduced does not introduce.
Step 5: trained ship detection model and monopod video camera are linked
By trained ship detection model transplantations to monopod video camera, each frame picture may be implemented in monopod video camera The arithmetic speed all detected, but the case where might have error detection or detection object scale error occur.
In order to solve the problems, such as accuracy, using coordinate information issue monopod video camera to monopod video camera act in the presence of Between the characteristics of being spaced, do not go to send each frame detection information, and be changed to send for each second it is primary, to collect the institute within this second The coordinate information and dimension information for all detection frames collected, count and filter out most believable coordinate and cloud is issued in size preparation Platform video camera.
Monopod video camera acts incoherent problem in order to prevent at this time, can according to the direction of motion of ship target and speed Letter coordinate transformation is to predict credible coordinate and ship target size and be sent to monopod video camera by serial ports, monopod video camera according to The information action received is realized from motion tracking and zoom.
In above-mentioned steps five, the method for the most credible coordinate of the screening and size is:
Present frame records coordinate (continuous record 20-25 frame of the ship target in entire picture at this time if there are object ship Target's center's point coordinates), often get 20 groups of data, data processings just carried out to this 20 groups of data, will most believable coordinate Point information is sent to monopod video camera by serial ports, and monopod video camera uses 3D algorithms, believes according to the most credible coordinate received The rotation of the length and width dimensions information of breath and ship target and zoom, monopod video camera, which remains, at this time is presented in display main bit stream Ship object width account for one third of picture width or so, and keep placed in the middle, and lasting tracking.
Artificial intelligence application on observation tracing task of the sea to ship, is exactly being made up work by this method using equipment Personnel generate sleepy sense due to staring at similar picture for a long time, or require staff to be found from one section of playing back videos and want to search What period the object of rope appears at, and similar needs the mankind to think deeply judgement but in very not humane work.
It is emphasized that embodiment of the present invention is illustrative, without being restrictive, therefore packet of the present invention Include the embodiment being not limited to described in specific implementation mode, it is every by those skilled in the art according to the technique and scheme of the present invention The other embodiment obtained, also belongs to the scope of protection of the invention.

Claims (2)

1. a kind of ship target real-time detection method based on monopod video camera, it is characterised in that:Include the following steps:
Step 1: including the video flowing of ship target by monopod video camera intake
The video stream packets of intake contain main bit stream and auxiliary code stream:
(1) main bit stream of rtsp video flowings for clearly indicating real-time video in the display;
(2) the auxiliary code stream of rtsp video flowings is handled as the input video stream of ship detection, and carries out target detection;
Step 2: training data pre-processes
The source of training data picture includes two parts:
A part of image credit is extracted in by the picture for containing ship target from main bit stream, that is, is used after obtaining rtsp video flowings The method batch of image procossing compresses picture and is changed resolution processes, ensures that key message is not lost;
Another part picture then comes from the ship target in extraction common data sets such as Pascal VOC, arranges the portion of needs Point, this part is mainly used for improving the generalization of ship detection, the recognition detection for ship target after furthering to zoom;
The above-mentioned picture classification handled well is arranged, ensures distal end ship type Small object in training set, various types of STOWAGE PLAN piece and possibility Error detection picture account for the certain ratio of training data;
Step 3: determining training pattern and improving the calculating speed of trained ship detection model
The darknet frame structures in linux system are applied in selection, facilitate the later stage using c/c++ language in embedded device On transplanting and docked with monopod video camera, train ship detection model using the principle of yolo;
Image is RGB triple channel images in auxiliary code stream in the step 1;
The method for improving detection speed is, using one type objects of training ship target, and using the volume base nuclear subsitution of multiple 1*1 sizes The principle of the convolution kernel of one n*n, meanwhile, auxiliary code stream input image size is changed by image processing algorithm, and ensure key Information is not lost, and calculative pixel number is reduced, and the method by reducing calculation amount improves calculating speed;
Step 4: being trained model adjustment according to actual conditions
Training data strictly screens, and meets the error detection factor that specific environment ship detection and this environment may introduce and does not draw Enter;
Step 5: trained ship detection model and monopod video camera are linked
By trained ship detection model transplantations to monopod video camera, the problem of in order to improve accuracy, counts and screen Go out most believable coordinate and monopod video camera is issued in size preparation.
2. the ship target real-time detection method according to claim 1 based on monopod video camera, it is characterised in that:The step In rapid five, the method for screening most credible coordinate and size is:
Present frame records the coordinate (mesh of continuous record 20-25 frame of the ship target in entire picture at this time if there are object ship Mark center point coordinate), 20 groups of data are often got, data processing just is carried out to this 20 groups of data, it will most believable coordinate points letter Breath is sent to monopod video camera by serial ports, monopod video camera using 3D algorithms, according to the most credible coordinate information received and The length and width dimensions information of ship target rotates and zoom, and monopod video camera remains the ship being presented in display main bit stream at this time Object width accounts for one third of picture width or so, and keeps placed in the middle, and lasting tracking.
CN201810173195.8A 2018-03-02 2018-03-02 A kind of ship target real-time detection method based on monopod video camera Pending CN108471497A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810173195.8A CN108471497A (en) 2018-03-02 2018-03-02 A kind of ship target real-time detection method based on monopod video camera

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810173195.8A CN108471497A (en) 2018-03-02 2018-03-02 A kind of ship target real-time detection method based on monopod video camera

Publications (1)

Publication Number Publication Date
CN108471497A true CN108471497A (en) 2018-08-31

Family

ID=63264084

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810173195.8A Pending CN108471497A (en) 2018-03-02 2018-03-02 A kind of ship target real-time detection method based on monopod video camera

Country Status (1)

Country Link
CN (1) CN108471497A (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109934088A (en) * 2019-01-10 2019-06-25 海南大学 Sea ship discrimination method based on deep learning
CN110278417A (en) * 2019-07-25 2019-09-24 上海莫吉娜智能信息科技有限公司 Monitoring device method for rapidly positioning and system based on millimetre-wave radar
CN110376593A (en) * 2019-08-05 2019-10-25 上海埃威航空电子有限公司 A kind of target apperception method and device based on laser radar
CN110427030A (en) * 2019-08-08 2019-11-08 上海大学 A kind of unmanned boat based on Tiny-YOLOship algorithm of target detection independently docks recovery method
CN111476163A (en) * 2020-04-07 2020-07-31 浙江大华技术股份有限公司 High-altitude parabolic monitoring method and device and computer storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1186923C (en) * 2003-04-03 2005-01-26 上海交通大学 Abnormal object automatic finding and tracking video camera system
CN102231798A (en) * 2011-06-24 2011-11-02 天津市亚安科技电子有限公司 Method for controlling PTZ (Pan/Tilt/Zoom) camera to zoom automatically and system thereof
CN104217215A (en) * 2014-08-28 2014-12-17 哈尔滨工程大学 Classification and identification method for foggy water surface image and clear water surface image
CN205179256U (en) * 2015-12-02 2016-04-20 北京视酷伟业科技股份有限公司 Automatic follow tracks of video linked system of boats and ships
CN106372590A (en) * 2016-08-29 2017-02-01 江苏科技大学 Sea surface ship intelligent tracking system and method based on machine vision
KR101737430B1 (en) * 2015-11-06 2017-05-19 주식회사 지오멕스소프트 A method of detecting objects in the image with moving background
CN107563387A (en) * 2017-09-14 2018-01-09 成都掌中全景信息技术有限公司 Frame method is selected in a kind of image object detection based on Recognition with Recurrent Neural Network

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1186923C (en) * 2003-04-03 2005-01-26 上海交通大学 Abnormal object automatic finding and tracking video camera system
CN102231798A (en) * 2011-06-24 2011-11-02 天津市亚安科技电子有限公司 Method for controlling PTZ (Pan/Tilt/Zoom) camera to zoom automatically and system thereof
CN104217215A (en) * 2014-08-28 2014-12-17 哈尔滨工程大学 Classification and identification method for foggy water surface image and clear water surface image
KR101737430B1 (en) * 2015-11-06 2017-05-19 주식회사 지오멕스소프트 A method of detecting objects in the image with moving background
CN205179256U (en) * 2015-12-02 2016-04-20 北京视酷伟业科技股份有限公司 Automatic follow tracks of video linked system of boats and ships
CN106372590A (en) * 2016-08-29 2017-02-01 江苏科技大学 Sea surface ship intelligent tracking system and method based on machine vision
CN107563387A (en) * 2017-09-14 2018-01-09 成都掌中全景信息技术有限公司 Frame method is selected in a kind of image object detection based on Recognition with Recurrent Neural Network

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
黄洁,姜志国等: "基于卷积神经网络的遥感图像舰船目标检测", 《北京航空航天大学学报》 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109934088A (en) * 2019-01-10 2019-06-25 海南大学 Sea ship discrimination method based on deep learning
CN110278417A (en) * 2019-07-25 2019-09-24 上海莫吉娜智能信息科技有限公司 Monitoring device method for rapidly positioning and system based on millimetre-wave radar
CN110376593A (en) * 2019-08-05 2019-10-25 上海埃威航空电子有限公司 A kind of target apperception method and device based on laser radar
CN110376593B (en) * 2019-08-05 2021-05-04 上海埃威航空电子有限公司 Target sensing method and device based on laser radar
CN110427030A (en) * 2019-08-08 2019-11-08 上海大学 A kind of unmanned boat based on Tiny-YOLOship algorithm of target detection independently docks recovery method
CN111476163A (en) * 2020-04-07 2020-07-31 浙江大华技术股份有限公司 High-altitude parabolic monitoring method and device and computer storage medium
CN111476163B (en) * 2020-04-07 2022-02-18 浙江大华技术股份有限公司 High-altitude parabolic monitoring method and device and computer storage medium

Similar Documents

Publication Publication Date Title
CN106096577B (en) A kind of target tracking method in camera distribution map
CN108471497A (en) A kind of ship target real-time detection method based on monopod video camera
CN103164706B (en) Object counting method and device based on video signal analysis
CN103413444B (en) A kind of traffic flow based on unmanned plane HD video is investigated method
CN104978567B (en) Vehicle checking method based on scene classification
CN108154110B (en) Intensive people flow statistical method based on deep learning people head detection
EP2956891B1 (en) Segmenting objects in multimedia data
CN109145708B (en) Pedestrian flow statistical method based on RGB and D information fusion
CN108334847A (en) A kind of face identification method based on deep learning under real scene
CN108197604A (en) Fast face positioning and tracing method based on embedded device
CN113762009B (en) Crowd counting method based on multi-scale feature fusion and double-attention mechanism
WO2020114116A1 (en) Pedestrian detection method based on dense crowds, and storage medium and processor
CN105160297A (en) Masked man event automatic detection method based on skin color characteristics
CN109063549A (en) High-resolution based on deep neural network is taken photo by plane video moving object detection method
CN104156978A (en) Multi-target dynamic following method based on balloon platform
CN109558790B (en) Pedestrian target detection method, device and system
Zhang et al. I-MMCCN: Improved MMCCN for RGB-T crowd counting of drone images
CN112464893A (en) Congestion degree classification method in complex environment
CN110866453B (en) Real-time crowd steady state identification method and device based on convolutional neural network
Wu et al. Vehicle Classification and Counting System Using YOLO Object Detection Technology.
Chen et al. Real-time garbage object detection with data augmentation and feature fusion using SUAV low-altitude remote sensing images
CN116311071A (en) Substation perimeter foreign matter identification method and system integrating frame difference and CA
CN112329550A (en) Weak supervision learning-based disaster-stricken building rapid positioning evaluation method and device
CN112258552A (en) Pedestrian multi-target tracking method under community monitoring scene
CN115880643A (en) Social distance monitoring method and device based on target detection algorithm

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
AD01 Patent right deemed abandoned
AD01 Patent right deemed abandoned

Effective date of abandoning: 20240419