CN102117326A - Traversal method used for searching image features - Google Patents

Traversal method used for searching image features Download PDF

Info

Publication number
CN102117326A
CN102117326A CN 201110047022 CN201110047022A CN102117326A CN 102117326 A CN102117326 A CN 102117326A CN 201110047022 CN201110047022 CN 201110047022 CN 201110047022 A CN201110047022 A CN 201110047022A CN 102117326 A CN102117326 A CN 102117326A
Authority
CN
China
Prior art keywords
register
feature extraction
data
buffer zone
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN 201110047022
Other languages
Chinese (zh)
Inventor
姜小波
周德祥
叶德盛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China University of Technology SCUT
Original Assignee
South China University of Technology SCUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT filed Critical South China University of Technology SCUT
Priority to CN 201110047022 priority Critical patent/CN102117326A/en
Publication of CN102117326A publication Critical patent/CN102117326A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Image Processing (AREA)

Abstract

The invention discloses a traversal method used for searching image features. The method adopts a hardware feature frame to traverse the image, wherein the hardware feature frame comprises a feature extraction frame and three buffer areas positioned at the upper, lower and right part of the feature extraction frame; and the traverse comprises repeating the flow consisting of a plurality of times of shift down operations, one time of right shift operation, a plurality of times of shift up operations and one time of shift right shift operation by more than one time, and then finishing traverse on the whole image. For the method, after each time of elementary operation, the elementary operation needed by the next time is determined fast according to the conditions of a state machine; the data is written in the corresponding buffer area while feature extraction is conducted in the feature extraction frame; and since the data write-in and the feature extraction are conducted at the same time, the steeping is less in most cases, and time of the feature extraction is longer than that of the data write-in, so that additional delay cannot be caused. The traversal method can effectively reduce hardware resources, and reduce the resources needed more effectively for the image having higher resolution ratio.

Description

A kind ofly be used for the traversal method that characteristics of image is searched
Technical field
The present invention relates to characteristics of image and search technical field, be specifically related to be used for the traversal method that characteristics of image is searched.
Background technology
Along with the increase to the man-machine interaction demand, people have proposed more and more higher requirement to man-machine interactive system.One of them very important index is exactly the resolution of system.But its inherent characteristics of man-machine interaction is that data processing amount is big, and real-time requires high.Here it is hinders the big bottleneck that it moves towards the high resolution system application.
Field of human-computer interaction has been used a large amount of mode identification methods.But the difference of both maximums is---the given width of cloth figure of pattern-recognition (is 20*20 as size), go to discern whether required target of looking for (as people's face) of this width of cloth figure again; And the given width of cloth figure of man-machine interaction (is 640*480 as size), whether go to discern where going up among this width of cloth figure again has required target of looking for (is people's face of 20*20 as size).Therefore, compare with pattern-recognition, man-machine interaction has had more a process of searching in the entire image traversal.
Specific objective (as people's face) searched to use feature.Feature is generally done by a plurality of pixels and is formed, and its pixel number difference that is comprised of different features.Therefore calculate number of calculations that each feature needs and also be operation time different, this is unfavorable for the realization of hardware.So generally all adopt method now based on the integrogram calculated characteristics.The value of each point is its upper left gray-scale value sum of having a few in the integrogram.Therefore when calculating each feature, only need to carry out 2 sub-addition computings and 1 subtraction gets final product with the integrated value of its 4 end points.So not only reduce computational complexity but also guaranteed each operation time.
Man-machine interaction is big because of its data processing amount, and the demanding characteristics of real-time become the big bottleneck that it is applied to common treatment.Recent years, also begun some man-machine interactions all over the world and handled the trial of accomplishing on FPGA or the ASIC.Consider factors such as computational complexity, arithmetic speed, power consumption, the way of main flow all adopts based on the feature of integrogram and searches and travel through now.Wherein, 3 kinds of typical methods are arranged.First method stores the integrogram of entire image in the register (register) into.Second method stores integrogram among the RAM into by row or by row.The third method, the capable buffer(such as the image that add specific quantity between RAM and register are 640*480, the size that detects target is 20*20, then adding (20+ stepping) individual length is 640 capable buffer), data are earlier from RAM to buffer, arrive register (referring to Proposed FPGA Hardware Architecture for High Frame Rate (〉 100fps) Face Detection Using Feature Cascade Classifiers---Hung-Chih Lai more at last, Marios Savvides, Tsuhan Chen Department of Electrical and Computer Engineering Carnegie Mellon University; FPGA-Based Face Detection System Using Haar Classifiers---Junguk Cho, Shahnam Mirzaei, Jason Oberg, Ryan Kastner Department of Computer Science and Engineering University of California).
No matter adopt above any design, in the middle of practical application, all can have restriction.First method, its required register number of using is a lot, can only be used for the smaller situation of entire image.Second method, the speed of extracting feature will be dragged the speed of slow total system too slowly.Handle though method three has been carried out compromise to the two kinds of methods in front, additive decrementation a lot of buffer resources.Present stage searches and travels through maximum difficult point based on the feature of integrogram and is: how to carry out fast as far as possible feature extraction with the least possible hardware resource.
Summary of the invention
The objective of the invention is to overcome the prior art above shortcomings, provide a kind of take resource little be used for the traversal method that characteristics of image is searched, can make characteristics of image search and travel through in speed and resource and reach good compromise on the two, concrete technical scheme is as follows.
A kind ofly be used for the traversal method that characteristics of image is searched, utilize the hardware characteristics frame that image is traveled through, described hardware characteristics frame comprises a feature extraction frame and is positioned at three upper and lower, right-hand buffer zones of feature extraction frame, described feature extraction frame is made up of the register array of a row * classify as M * N, M is the natural number greater than 2, and N is the natural number greater than 1; In described feature extraction frame, capable except that the top X, capable, the most right-hand Y row of below X, each register upwards links to each other with the register that apart from this register is X at upper and lower, on right, link to each other with the register that apart from this register is Y, X is the row stepping, the X value is the natural number less than M/2, and Y is the row stepping, and the Y value is the natural number less than N; Described upper and lower buffer zone is formed by the register array of row * classify as X * N; Right buffer zone is made up of the register array of row * classify as M * Y; The register that described each register and this register of below feature extraction frame middle distance of going up in the buffer zone is X links to each other, each register in the described down buffer zone and this register of top feature extraction frame middle distance are that the register of X links to each other, and each register in the right buffer zone and this register of left feature extraction frame middle distance are that the register of Y links to each other; Described traversal comprises that once above the repetition finish putting in order the traversal of sub-picture after repeatedly moving down operation, right-shift operation, moving the flow process that operation and right-shift operation once form on repeatedly.
In the above-mentioned traversal method, described moving down is operating as: in the described feature extraction frame except that the capable register of the top X, each register is write into the data of being stored in the register of its top distance for row stepping X, and the register in the described buffer zone is down write into the data of being stored in the register in the connected feature extraction frame.
In the above-mentioned traversal method, described right-shift operation is: in the described feature extraction frame except that the register of left Y row, each register is write into its left distance in the register of row stepping Y with the data of being stored, and the register in the described right buffer zone is write into the data of being stored in the register in the connected feature extraction frame.
In the above-mentioned traversal method, move on described and be operating as: in the described feature extraction frame except that the register that below X is capable, each register is write into the data of being stored in the register of its below distance for row stepping X, and the described register of going up in the buffer zone is write into the data of being stored in the register in the connected feature extraction frame.
In the above-mentioned traversal method, described feature extraction frame is the many input registers array that is used to store the integrogram data of M * N, described upper and lower buffer zone is single many Output Shift Registers of the input array that is used to store the integrogram data of X * N, and described right buffer zone is single many Output Shift Registers of the input array that is used to store the integrogram data of M * Y.
Above-mentioned traversal method specifically can comprise the steps:
When (1) traversal began, the data in the feature extraction frame all were invalid, carry out feature extraction and must extract frame to the most upper left data input feature vector of image earlier; Because feature extraction frame and periphery do not have interface, so must pass through buffer zone input data down earlier;
(2) data inputs buffer zone down once moves down operation then, two row of the below of data shift-in feature extraction frame;
(3) repeating step (2) is up to the just whole shift-in feature extraction frames of the most upper left data of image;
(4) carry out an image characteristics extraction because next time operation is also for moving down operation, so simultaneously image more down two capable view data deposit down buffer zone in, treat once to move down operation after this feature extraction is finished;
(5) repeating step (4) does not have data toward the below again until image.At this moment, being operating as next time moves to right, and right buffer zone is started working, and in feature extraction, image deposits right buffer zone in toward the view data of right-hand two row again;
(6) carry out right-shift operation one time, at this moment, being operating as next time moves, and last buffer zone is started working, and in feature extraction, the view data of two row deposits buffer zone in to image toward the top again;
(7) move operation on carrying out once, carry out image characteristics extraction again; Simultaneously, because operation next time moves operation on also being, the data data of two row deposit buffer zone in so image is again toward the top;
(8) repeating step (7) does not have data toward the top again until image; At this moment, being operating as next time moves to right, and right buffer zone is started working, and in feature extraction, image deposits right buffer zone in toward the view data of right-hand two row again;
(9) repeat step (4) before to step (8), finished by traversal until whole target image.
The feature of its required extraction of the present invention is provided by training result, can directly be solidified into the hardware line, also can utilize the combination of MUX and ROM to finish Feature Extraction.These two kinds of methods can realize with existing proven technique.
With respect to prior art, the present invention has following advantage:
When (1) being applied to higher resolution image, needed hardware resource seldom.With the image is 640*480, and the size that detects target is 20*20, and stepping is 2 as an illustration.Adopt the used scheme of the present invention to need 520 (20*20+3*2*20) registers.If adopt the scheme of whole integrogram storage to need 307200 (640*480) registers.If adopt adding the scheme of buffer, to need 400 (20*20) registers and 22 degree of depth be 640 block RAM.This shows, adopt scheme of the present invention can effectively reduce hardware resource, and, can reduce resource requirement effectively more for the high more image of resolution.
(2) when effectively reducing hardware resource, can big influence not arranged to processing speed.The present invention can make required next time basic operation rapidly according to the situation of state machine after basic operation each time.When in the feature extraction frame, extracting, data are write in the corresponding buffer zone (buffer zone is only arranged at every turn in work---write data).Because the two carries out simultaneously, and under most of situation, stepping is smaller, the time spent of feature extraction is longer than data and writes, and can't bring extra time-delay like this.And the situation that stepping is big and the very fast end of feature extraction is so relatively seldom, and in this case, though can bring certain time-delay, this also is complete acceptable compared with the saving of resource.
Description of drawings
Fig. 1 is the structured flowchart that is used for the hardware characteristics frame of traversing graph picture of the present invention, and reg represents register among the figure.
Move operation on Fig. 2 hardware characteristics frame, scheming medium and small square frame is register reg.
Fig. 3 is the operation that moves down of hardware characteristics frame, and scheming medium and small square frame is register reg.
Fig. 4 is the right-shift operation of hardware characteristics frame, and scheming medium and small square frame is register reg.
Fig. 5 is the hardware characteristics frame among the embodiment, and scheming medium and small square frame is register reg.
Fig. 6 is the procedure chart of entire image search plan.
Embodiment
Below in conjunction with accompanying drawing step of the present invention is further described, but the scope of protection of present invention is not limited to down the scope of example statement.
Fig. 1 is the structured flowchart of hardware characteristics frame, comprises a feature extraction frame and is positioned at three upper and lower, right-hand buffer zones of feature extraction frame; The feature extraction frame is made up of the register array that is used to store the integrogram data of a row * classify as M * N, and M is the natural number greater than 2, and N is the natural number greater than 1; In described register array, capable except that the top X, capable, the most right-hand Y row of below X, each register upwards links to each other with the register that apart from this register is X at upper and lower, on right, link to each other with the register that apart from this register is Y, X is the row stepping, X gets the natural number less than M/2, and Y is the row stepping, and Y gets the natural number less than N; Upper and lower buffer zone is formed by the register array that is used to store the integrogram data of row * classify as X * N; Right buffer zone is made up of the register array of row * classify as the storage integrogram data of M * Y; The register that described each register and this register of below feature extraction frame middle distance of going up in the buffer zone is X links to each other, each register in the described down buffer zone and this register of top feature extraction frame middle distance are that the register of X links to each other, and each register in the right buffer zone and this register of left feature extraction frame middle distance are that the register of Y links to each other.
As Fig. 2, move on the hardware characteristics frame and be operating as: in the described feature extraction frame except that the register that below X is capable, each register is write into the data of being stored in the register of its below distance for row stepping X, and the described register of going up in the buffer zone is write into the data of being stored in the register in the connected feature extraction frame.
As Fig. 3, moving down of hardware characteristics frame is operating as: in the described feature extraction frame except that the capable register of the top X, each register is write into the data of being stored in the register of its top distance for row stepping X, and the register in the described buffer zone is down write into the data of being stored in the register in the connected feature extraction frame.
As Fig. 4, the right-shift operation of hardware characteristics frame is: in the described feature extraction frame except that the register of left Y row, each register is write into its left distance in the register of row stepping Y with the data of being stored, and the register in the described right buffer zone is write into the data of being stored in the register in the connected feature extraction frame.
As shown in Figure 5, be the hardware characteristics frame of present embodiment, its feature extraction frame size is 10 * 10; In the above and below of feature extraction frame, be respectively the buffer zone of a row * behavior 10 * 2, be called buffer zone and following buffer zone; Right-hand at the feature extraction frame is the buffer zone of a row * behavior 2 * 10, is referred to as right buffer zone.
As shown in Figure 6, present embodiment entire image ergodic process is as follows:
When (1) traversal began, the data in the feature extraction frame all were invalid, carry out feature extraction and must extract frame to the most upper left data input feature vector of image earlier; Because feature extraction frame and periphery do not have interface, so must pass through buffer zone input data down earlier;
(2) data inputs buffer zone down once moves down operation then, two row of the below of data shift-in feature extraction frame;
(3) repeating step (2), 5 times altogether, up to the just whole shift-in feature extraction frames of the most upper left data of image;
(4) carry out an image characteristics extraction because next time operation is also for moving down operation, so simultaneously image more down two capable view data deposit down buffer zone in, treat once to move down operation after this feature extraction is finished;
(5) repeating step (4) does not have data toward the below again until image.At this moment, being operating as next time moves to right, and right buffer zone is started working, and in feature extraction, image deposits right buffer zone in toward the view data of right-hand two row again;
(6) carry out right-shift operation one time, at this moment, being operating as next time moves, and last buffer zone is started working, and in feature extraction, the view data of two row deposits buffer zone in to image toward the top again;
(7) move operation on carrying out once, carry out image characteristics extraction again; Simultaneously, because operation next time moves operation on also being, the data data of two row deposit buffer zone in so image is again toward the top;
(8) repeating step (7) does not have data toward the top again until image; At this moment, being operating as next time moves to right, and right buffer zone is started working, and in feature extraction, image deposits right buffer zone in toward the view data of right-hand two row again;
(9) repeat step (4) before to step (8), as shown in Figure 6, finished by traversal until whole target image.
When the present invention was applied to higher resolution image, needed hardware resource seldom.With the image is 640*480, and the size that detects target is 20*20, and stepping is 2 to be example.Adopt the used scheme of the present invention to need 520 (20*20+3*2*20) registers.If adopt the scheme of whole integrogram storage to need 307200 (640*480) registers.If adopt adding the scheme of buffer, to need 400 (20*20) registers and 22 degree of depth be 640 block RAM.This shows, adopt scheme of the present invention can effectively reduce hardware resource, and, can reduce resource requirement effectively more for the high more image of resolution.

Claims (6)

1. one kind is used for the traversal method that characteristics of image is searched, utilize the hardware characteristics frame that image is traveled through, it is characterized in that described hardware characteristics frame comprises a feature extraction frame and is positioned at three upper and lower, right-hand buffer zones of feature extraction frame, described feature extraction frame is made up of the register array of a row * classify as M * N, M is the natural number greater than 2, and N is the natural number greater than 1; In described feature extraction frame, capable except that the top X, capable, the most right-hand Y row of below X, each register upwards links to each other with the register that apart from this register is X at upper and lower, on right, link to each other with the register that apart from this register is Y, X is the row stepping, the X value is the natural number less than M/2, and Y is the row stepping, and the Y value is the natural number less than N; Described upper and lower buffer zone is formed by the register array of row * classify as X * N; Right buffer zone is made up of the register array of row * classify as M * Y; The register that described each register and this register of below feature extraction frame middle distance of going up in the buffer zone is X links to each other, each register in the described down buffer zone and this register of top feature extraction frame middle distance are that the register of X links to each other, and each register in the right buffer zone and this register of left feature extraction frame middle distance are that the register of Y links to each other; Described traversal comprises that once above the repetition finish putting in order the traversal of sub-picture after repeatedly moving down operation, right-shift operation, moving the flow process that operation and right-shift operation once form on repeatedly.
2. traversal method according to claim 1, it is characterized in that described moving down is operating as: in the described feature extraction frame except that the capable register of the top X, each register is write into the data of being stored in the register of its top distance for row stepping X, and the register in the described buffer zone is down write into the data of being stored in the register in the connected feature extraction frame.
3. traversal method according to claim 1, it is characterized in that described right-shift operation is: in the described feature extraction frame except that the register of left Y row, each register is write into its left distance in the register of row stepping Y with the data of being stored, and the register in the described right buffer zone is write into the data of being stored in the register in the connected feature extraction frame.
4. traversal method according to claim 1, it is characterized in that moving on described and be operating as: in the described feature extraction frame except that the register that below X is capable, each register is write into the data of being stored in the register of its below distance for row stepping X, and the described register of going up in the buffer zone is write into the data of being stored in the register in the connected feature extraction frame.
5. as each described traversal method of claim 1~4, it is characterized in that described feature extraction frame is the many input registers array that is used to store the integrogram data of M * N, described upper and lower buffer zone is single many Output Shift Registers of the input array that is used to store the integrogram data of X * N, and described right buffer zone is single many Output Shift Registers of the input array that is used to store the integrogram data of M * Y.
6. traversal method according to claim 5 is characterized in that specifically comprising the steps:
When (1) traversal began, the data in the feature extraction frame all were invalid, carry out feature extraction and must extract frame to the most upper left data input feature vector of image earlier; Because feature extraction frame and periphery do not have interface, so must pass through buffer zone input data down earlier;
(2) data inputs buffer zone down once moves down operation then, two row of the below of data shift-in feature extraction frame;
(3) repeating step (2) is up to the just whole shift-in feature extraction frames of the most upper left data of image;
(4) carry out an image characteristics extraction because next time operation is also for moving down operation, so simultaneously image more down two capable view data deposit down buffer zone in, treat once to move down operation after this feature extraction is finished;
(5) repeating step (4) does not have data toward the below again until image; At this moment, being operating as next time moves to right, and right buffer zone is started working, and in feature extraction, image deposits right buffer zone in toward the view data of right-hand two row again;
(6) carry out right-shift operation one time, at this moment, being operating as next time moves, and last buffer zone is started working, and in feature extraction, the view data of two row deposits buffer zone in to image toward the top again;
(7) move operation on carrying out once, carry out image characteristics extraction again; Simultaneously, because operation next time moves operation on also being, the data data of two row deposit buffer zone in so image is again toward the top;
(8) repeating step (7) does not have data toward the top again until image; At this moment, being operating as next time moves to right, and right buffer zone is started working, and in feature extraction, image deposits right buffer zone in toward the view data of right-hand two row again;
(9) repeat step (4) before to step (8), finished by traversal until whole target image.
CN 201110047022 2011-02-28 2011-02-28 Traversal method used for searching image features Pending CN102117326A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201110047022 CN102117326A (en) 2011-02-28 2011-02-28 Traversal method used for searching image features

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201110047022 CN102117326A (en) 2011-02-28 2011-02-28 Traversal method used for searching image features

Publications (1)

Publication Number Publication Date
CN102117326A true CN102117326A (en) 2011-07-06

Family

ID=44216096

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201110047022 Pending CN102117326A (en) 2011-02-28 2011-02-28 Traversal method used for searching image features

Country Status (1)

Country Link
CN (1) CN102117326A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102867181A (en) * 2012-07-27 2013-01-09 华南理工大学 Characteristic extraction module for digital image processing and traversing method
CN103546752A (en) * 2013-10-15 2014-01-29 华南理工大学 Image size compression traversing method on basis of hardware parallel architecture

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100111446A1 (en) * 2008-10-31 2010-05-06 Samsung Electronics Co., Ltd. Image processing apparatus and method
US20100128993A1 (en) * 2008-11-21 2010-05-27 Nvidia Corporation Application of classifiers to sub-sampled integral images for detecting faces in images
CN101794435A (en) * 2010-03-04 2010-08-04 成都三泰电子实业股份有限公司 Binary image noise-reduction method based on integral graph and binary image processing system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100111446A1 (en) * 2008-10-31 2010-05-06 Samsung Electronics Co., Ltd. Image processing apparatus and method
US20100128993A1 (en) * 2008-11-21 2010-05-27 Nvidia Corporation Application of classifiers to sub-sampled integral images for detecting faces in images
CN101794435A (en) * 2010-03-04 2010-08-04 成都三泰电子实业股份有限公司 Binary image noise-reduction method based on integral graph and binary image processing system

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102867181A (en) * 2012-07-27 2013-01-09 华南理工大学 Characteristic extraction module for digital image processing and traversing method
CN103546752A (en) * 2013-10-15 2014-01-29 华南理工大学 Image size compression traversing method on basis of hardware parallel architecture
CN103546752B (en) * 2013-10-15 2016-10-05 华南理工大学 A kind of picture size based on hardware concurrent framework compression traversal method

Similar Documents

Publication Publication Date Title
CN107808131A (en) Dynamic gesture identification method based on binary channel depth convolutional neural networks
CN103743750B (en) A kind of generation method of distribution diagram of surface damage of heavy calibre optical element
CN104463873A (en) Image target repositioning method based on local uniform scaling
CN103218600A (en) Real-time face detection algorithm
CN105139342A (en) Method and device for zooming pictures
CN102117409B (en) Hardware feature box for searching and traversing feathers based on integrogram
CN105046632A (en) Space-time efficient binary image binary logic operation method
Liao et al. Residual attention unit for action recognition
CN102117326A (en) Traversal method used for searching image features
CN202736078U (en) Feature extraction module used for digital image processing
CN102867181A (en) Characteristic extraction module for digital image processing and traversing method
CN106802787A (en) MapReduce optimization methods based on GPU sequences
Liang et al. MEANet: An effective and lightweight solution for salient object detection in optical remote sensing images
CN112164065A (en) Real-time image semantic segmentation method based on lightweight convolutional neural network
Kumawat et al. ShuffleBlock: Shuffle to regularize deep convolutional neural networks
Yang et al. Scinet: Semantic cue infusion network for lane detection
CN106920264A (en) A kind of target detection hardware configuration based on deformable member model algorithm
CN109583584A (en) The CNN with full articulamentum can be made to receive the method and system of indefinite shape input
Hu et al. Weakly supervised instance segmentation using multi-stage erasing refinement and saliency-guided proposals ordering
CN112329544A (en) Gesture recognition machine learning method and system based on depth information
Runyu Pedestrian detection based on SENet with attention mechanism
Zong et al. Deep feature filtering and contextual information gathering network for RGB-d salient object detection
CN104881119B (en) Video input-based air gesture trail acquiring method
CN106934818A (en) A kind of hand exercise tracking and system
Li et al. Global information progressive aggregation network for lightweight salient object detection

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C02 Deemed withdrawal of patent application after publication (patent law 2001)
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20110706