US7693301B2 - Known face guided imaging method - Google Patents
Known face guided imaging method Download PDFInfo
- Publication number
- US7693301B2 US7693301B2 US11/545,423 US54542306A US7693301B2 US 7693301 B2 US7693301 B2 US 7693301B2 US 54542306 A US54542306 A US 54542306A US 7693301 B2 US7693301 B2 US 7693301B2
- Authority
- US
- United States
- Prior art keywords
- face
- scale
- searching
- sensed
- known face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/167—Detection; Localisation; Normalisation using comparisons between temporally consecutive images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
Definitions
- the present invention relates to a known face guided imaging method, and more particularly to a known face guided imaging method capable of reducing the huge quantity of computing values required for a face detection and effectively enhancing the speed and efficiency of the face detection.
- the algorithm for detecting a face has been disclosed many publications, and the most popular one is the face detector designed according to a Gentle Adaboost (GAB) algorithm, and the face detector uses a Haar-like feature to identify a face and a specific quantity of face pattern samples to train a required face classifier to determine which image of the scene belongs to (or not belongs) to a face, so as to detect a face in the image and provide a quick identification.
- GAB Gentle Adaboost
- the GAB algorithm chooses the best Haar feature having the minimum weighted error e m from all features. For each weak classifier ⁇ m (x), the GAB algorithm chooses a feature j, and the error function is minimized by Equation 1:
- Equation 1 show that although the GAB algorithm updates the classifier at each stage by a confidence-rated real value, the misclassification error defined in the GAB algorithm is discrete.
- ⁇ i is a Boolean variable which is equal to 1 if v i is computed for misclassification, and equal to 0 if ⁇ i is computed for classification.
- a weak classifier with a binary output in the discrete Adaboost algorithm cannot show a Haar-like feature being situated at a very good distribution status.
- the misclassification error defined in the algorithm is insufficient to describe the distribution status of the misclassification error precisely.
- the inventor of the present invention redefines the misclassification error em in the GAB algorithm as follows:
- ⁇ i stands for the distance between a confidence-rated real value and an expected class label.
- ⁇ i stands for the distance between a confidence-rated real value and an expected class label.
- the two histogram bins show the difference of two defined equations, wherein positive samples in the histogram bins have different distributions for the features i and j. For simplicity the negative samples own sample distribution for feature i and j. If Equation 1 is used, then the resultant error summation will be the same for the two types of feature spaces. If Equation 2 is used, the resultant error summation of the feature will be smaller than the computed result of the feature i. As to a greedy searching scheme, feature j is selected to build the weak classifier according to the definition of a weak classifier function of the weak classifier.
- the output confidence value will be close to zero, or else the output confidence value will be close to 1 or ⁇ 1.
- the result obviously shows that the output confidence value of the feature j is much larger than the output confidence value of the feature i.
- the sample in the space of feature j can be separated from the space of feature i more easily, and thus the definition of a confidence-rated misclassification error becomes more reasonable.
- the Haar-like feature is defined in its feature pool.
- Four basic units are used for detecting an object in an image window as shown in FIG. 2 , wherein the prototypes 10 , 11 represent edge features; the prototype 12 represents a line feature; the prototype 13 represents a special diagonal line feature; the black area represents a negative weight; and the white area represents a positive weight.
- the inventor of the present invention also discloses a way of defining a Haar-like feature according to the foregoing algorithm to separate the samples in the histogram bin more easily, and the Haar-like feature for detecting an object in an image window is defined by eight basic units as shown in FIG.
- the feature prototypes 20 , 21 represent edge features; the black area represents a negative weight; the white area represents a positive weight; and the black and white areas are distributed on the same horizontal line or vertical line and separated with each other by a certain specific distance.
- the feature prototypes 22 , 23 represent diagonal features; the black area represents a negative weight; the white area represents a positive weight; and the diagonals of the black and white areas are perpendicular with each other.
- the feature prototypes 24 , 25 also represent diagonal feature prototypes; the black area represents a negative weight; the white area represents a positive weight; and the diagonals of the black and white areas are parallel with each other.
- the prototypes 26 , 27 represent special diagonal line features; the black area represents a negative weight; and the white area represents a positive weight, wherein 1 ⁇ 4 of the areas of the black area and the white area are overlapped along their diagonal directions.
- a face detector uses ten different sized searching windows to search every preview image and adjusts the size of the searching windows one by one and moves the searching windows horizontally and vertically to search a face repeatedly, and thus the computing values for such detection process is huge, and the speed and efficiency of the face detection are very low, and the performance cannot meet consumers' requirements and expectations.
- Such arrangement simply sends a corresponding image patch within the scale range to a face detector for the face detection, and thus the searching space will be constrained in a small group of scale ranges without the need of detecting the whole image with different scales within the scale range, and it also effectively reduces the huge quantity of computing values required for the detection process and greatly enhances the speed and efficiency of the face detection.
- Another objective of the present invention is to build a label mask for completing the scale constraint, and the label mask has the same scale as the search image, and the area of the boundary box of the corresponding known face and the area of the remaining of the unknown face are labeled with different values, such that if the center of the searching windows falls in an area labeled for unknown faces during a face detection process, the scale constraint of the unknown face will be added. If the center of the searching window falls in an area labeled for known faces, the scale constraint of the known face will be applied.
- Another objective of the present invention is to define the range of the scale constraint of the searching windows based on the range of the known face area and unknown face area on the label mask.
- FIG. 1 is a schematic view of the distributions of the feature i and the feature j by a traditional histogram bin method
- FIG. 2 is a schematic view of four basic units used for defining feature prototypes used in traditional Haar-like feature
- FIG. 3 is a schematic view of eight basic units used for defining feature prototypes used in the inventor's related patent applications;
- FIG. 4 is a schematic view of relative positions of a searching window and a boundary box on a preview image
- FIG. 5 is a flow chart of a preferred embodiment of the present invention.
- FIG. 6 is a schematic view of two known faces existed in a preview image according to a preferred embodiment as depicted in FIG. 5 ;
- FIG. 7 is a schematic view of a label mask built by a preview image as depicted in FIG. 6 .
- the present invention discloses a known face guided imaging method, and the method is applied for an electronic imaging device, such that if the electronic imaging device is switched to a preview state, a detection and tracking module embedded in the electronic imaging device will be used to define a plurality of searching windows 40 in different scales as shown in FIG. 4 for adjusting the scale of the searching windows 40 one by one and moving the searching window in the whole preview image 41 horizontally and vertically to search a face repeatedly. As long as a face is searched as shown in FIG. 4 , its position and scale are labeled by a boundary box 42 .
- the detection and tracking module will perform a tracking to the face synchronously to update the position and scale of the boundary box 42 , so as to accurately position the correct position of the face in the preview image 41 and achieve the correct auto focus and auto exposure of the electronic imaging device. Further, the white balance and color transfer of the preview image 41 can be adjusted correctly by the face image.
- the present invention adds a scale constraint to the searching window 40 to expedite the search and detection speed, when the searching window 40 searches a face for a preview image 41 horizontally and vertically, and the method bases on the scale of a sensed known face of the previous frame to define the scale range of the searching window 40 , and just sends an image patch corresponding to the scale range in the current frame to a face detector when a face searching loop is performed horizontally and vertically to the current frame for processing the face detection.
- the searching space is constrained within a small group of scale ranges without the need of searching the ranges of various different scales in the whole preview image 41 , and it effectively reduces the huge quantity of computing values required by the detection process, and greatly enhances the speed and efficiency for the face detection.
- the present invention bases on a face classifier built by the Adaboost algorithm to control the discrimination ability of the face easily by a number of layers of a cascaded structure of the face classifier.
- the present invention bases on the foregoing Adaboost algorithm to train a face classifier having ten layers, such that after a series of face samples are classified, the false alarm rate of the face classifier is very low and most suitable for detecting unknown faces in an image.
- the face classifier having seven layers trained by the present invention performs detections and tracking for the known faces detected in the previous frame according to the following procedure and as shown in FIG. 5 .
- the known faces detected in the previous frame are detected and tracked.
- the detection and tracking procedure comprises the following steps:
- x l/20
- 20 is the width (number of pixels) of the minimum scale of a searching window 61 and s is a scale factor
- l is the width (number of pixels) of a boundary box of the known face in the previous frame.
- the scale range of the searching window 61 is defined by the following two methods:
- the searching windows 61 of all scales must be checked.
- x max l max /20, as shown in FIG. 6
- the scale of the searching window 61 uses 20 ⁇ 20 as a base to adjust the scale in the range from 1.45 times to 6 times and the scale is increased by 1.2 times for every searching loop, and thus it only requires to search the plurality of known faces in the current frame by using the searching windows 61 of seven scales such as 34.6 ⁇ 34.6, 41.5 ⁇ 41.5, 49.8 ⁇ 49.8, 59.7 ⁇ 59.7, 71.7 ⁇ 71.7, 86 ⁇ 86, 103.2 ⁇ 103.2 pixels one by one. Therefore, if the face detector detects a new face in the current frame, the scale of the searching window 61 of the searched new face should not be less than x max /(s) 4 .
- 0 ⁇ 1, 0 ⁇ 1 and b i is used for computing a changing tendency in an image, and ⁇ and ⁇ are constants.
- the present invention uses the searching windows to perform a face searching loop horizontally and vertically for the current frame, and the searching space is constrained in a small group of scale ranges according to the scale of the sensed known face in the previous frame, without the need of detecting the search range of various scales in the whole search image.
- the invention not only effectively reduces the huge quantity of computing values required for the detection process and greatly enhances the speed and efficiency of the face detection, such that the electronic imaging device can quickly and accurately position the correct position of a face in the preview image, and quickly and accurately achieve the advanced functions such as the auto focus and auto exposure for taking high-quality images.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
Description
A stage of Haar feature classifier construction |
using GAB |
1. | Start with weights wi = 1/2p and 1/2l where p and l are the number of |
positive and negatives class samples. | |
2. | Repeat for m = 1, 2, . . . , M. |
(a) | For each Haar feature j, fm(x) = Pw(y = 1|x) − Pw(y = −1|x) | |
using only the feature j values. | ||
(b) | Choose the best feature confidence set of values fm(x) giving the | |
minimum weighted error em = Ew [1(y |
||
(c) | Update F(x) ← F(x) + fm(x) | |
(d) | Set wi ← wi exp [−yi, fm(xi)], i = 1, 2, . . . , N, and renormalize | |
so that
|
||
3. | Output the classifier sign [F(x)] = sign
|
wi is a sample weight.
- (4000) Capture a preview image.
- (4010) Downsize the scale of a preview image to a search image 60 of 160×120 pixels as shown in
FIG. 6 ; - (4020) Build a label mask 70 as shown in
FIGS. 6 and 7 to assist the completion of a scale constraint, and the label mask 70 has the same scale of the search image 60, and the values are 0, 1 2, and so on, the shaded square area (which is the boundary box) of the label mask 70 as shown inFIG. 7 indicates the position and scale of the known face in the previous frame, and each pixel is labeled as 1 and 2 separately, and the remaining pixels are labeled as zero to represent the portion of the unknown face, and a searching window 61 moves horizontally and vertically to search a face in the search image 60 during a detection process as shown inFIG. 6 . In a preferred embodiment of the present invention, the minimum scale of the searching window 61 is a square of 20×20 pixels, and the scale will be amplified by 1.2 times for every searching loop such as 20×20, 24×24, 28.8×28.8, 34.6×34.6, and so one. If the center of the searching window 61 falls in an area labeled as 0 which is equivalent to the position 71 as shown inFIG. 7 , then a scale constraint for the unknown face will be added. If the center of the searching window 61 falls in an area labeled as non-zero, then a scale constraint of the known face will be applied; - (4030) Compute the scale ranges of the searching window 61, and the scale range defines a scale constraint of the searching window according to the range and positions of the pixels of each known face and unknown face on the label mask 70. In a preferred embodiment of the present invention, the scale range for a known face is defined as:
=└x/(s)2 , x*(s)2┘ (3)
=[1,6] (4)
=└x max/(s)4,6┘ (5)
- (4040) Select a scale according to the computed scale range.
- (4050) Set the scale of the searching
window 61 according to the selected scale as shown inFIG. 6 , and let the searchingwindow 61 move horizontally and vertically on thewhole search image 60 to search a known face or an unknown face. - (4060) Determine whether or not the center of the searching
window 61 falls within an area labeled as 0 on thelabel mask 70 as shown inFIGS. 6 and 7 ; if yes, then adopt the scale constraint of unknown face, and continue Step (4070); or determine whether or not the center of the searching window falls in an area labeled as non-zero; if yes, then adopt the scale constraint of the known face, and continue Step (4070); or else continue Step (4080). - (4070) Send an image patch corresponding to the scale range of the current frame to the face detector to perform a face detection.
- (4080) Determine whether or not the searching
window 61 has searched thewhole search image 60 horizontally and vertically to the utmost rear end; if yes, then continue Step (4090); or else, continue Step (4050). - (4090) Determine whether or not all scales in the scale range have been selected and used; if yes, continue Step (4100); or else continue Step (4040), and select the next scale according to the computed scale range.
- (4100) Use an overlapping method to build an association between a newly detected known face in the current frame and a detected known face in the previous frame; and build an association between the detected known faces before and after the detection by defining a searching area around the old position in the known face in the previous frame, and the searching area is a bounding rectangle of the known face. In the searching area, the position having the most overlapped portion with the known face is considered as a new position of the known face; and if no face is detected, then the originally detected known face will be deleted.
- (4110) In the present invention, the position and scale of the detected face is labeled in a
boundary box 62 as shown inFIG. 6 , and theboundary box 62 updates each image according to the detected result of the face detector, and thus the scale of theboundary box 62 is always not a constant, and the present invention uses an exponential smoothing method to make a smoother change to the scale of theboundary box 62 or uses a double exponential smoothing method to make a tendentious change to the boundary box. In this preferred embodiment, if the width yi of a detected face in an image i is computed according to the double exponential smoothing, and its smoothed width Si is defined as:
S i =αy i+(1−α)(S i-1 +b i-1) (6)
b i=γ(S i −S i-1)+(1−γ)b i-1 (7)
b i =y 2 −b 1 (9)
Claims (10)
R=[x/(s)2,x*(s)2],
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/545,423 US7693301B2 (en) | 2006-10-11 | 2006-10-11 | Known face guided imaging method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/545,423 US7693301B2 (en) | 2006-10-11 | 2006-10-11 | Known face guided imaging method |
Publications (2)
Publication Number | Publication Date |
---|---|
US20080089560A1 US20080089560A1 (en) | 2008-04-17 |
US7693301B2 true US7693301B2 (en) | 2010-04-06 |
Family
ID=39321404
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/545,423 Active 2029-02-04 US7693301B2 (en) | 2006-10-11 | 2006-10-11 | Known face guided imaging method |
Country Status (1)
Country | Link |
---|---|
US (1) | US7693301B2 (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090018985A1 (en) * | 2007-07-13 | 2009-01-15 | Microsoft Corporation | Histogram-based classifiers having variable bin sizes |
US20090116705A1 (en) * | 2007-11-01 | 2009-05-07 | Sony Corporation | Image processing apparatus, image processing method, image processing program, image capturing apparatus, and controlling method thereof |
US20100008549A1 (en) * | 2008-07-08 | 2010-01-14 | Clay Jessen | Increasing Face Detection Speed |
US20100194903A1 (en) * | 2009-02-03 | 2010-08-05 | Kabushiki Kaisha Toshiba | Mobile electronic device having camera |
US20110293173A1 (en) * | 2010-05-25 | 2011-12-01 | Porikli Fatih M | Object Detection Using Combinations of Relational Features in Images |
US20130089235A1 (en) * | 2011-10-06 | 2013-04-11 | Samsung Electronics Co., Ltd. | Mobile apparatus and method for controlling the same |
Families Citing this family (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5044321B2 (en) * | 2006-09-13 | 2012-10-10 | 株式会社リコー | Imaging apparatus and subject detection method |
KR101330636B1 (en) * | 2007-01-24 | 2013-11-18 | 삼성전자주식회사 | Face view determining apparatus and method and face detection apparatus and method employing the same |
US8031970B2 (en) * | 2007-08-27 | 2011-10-04 | Arcsoft, Inc. | Method of restoring closed-eye portrait photo |
US8855360B2 (en) * | 2008-07-23 | 2014-10-07 | Qualcomm Technologies, Inc. | System and method for face tracking |
CN101894262B (en) * | 2009-05-20 | 2014-07-09 | 索尼株式会社 | Method and apparatus for classifying image |
US8737725B2 (en) * | 2010-09-20 | 2014-05-27 | Siemens Aktiengesellschaft | Method and system for learning based object detection in medical images |
US8965046B2 (en) | 2012-03-16 | 2015-02-24 | Qualcomm Technologies, Inc. | Method, apparatus, and manufacture for smiling face detection |
US10846838B2 (en) * | 2016-11-25 | 2020-11-24 | Nec Corporation | Image generation device, image generation method, and storage medium storing program |
CN109033924A (en) * | 2017-06-08 | 2018-12-18 | 北京君正集成电路股份有限公司 | The method and device of humanoid detection in a kind of video |
US10810255B2 (en) * | 2017-09-14 | 2020-10-20 | Avigilon Corporation | Method and system for interfacing with a user to facilitate an image search for a person-of-interest |
CN108764139B (en) * | 2018-05-29 | 2021-01-29 | Oppo(重庆)智能科技有限公司 | Face detection method, mobile terminal and computer readable storage medium |
JP7200965B2 (en) * | 2020-03-25 | 2023-01-10 | カシオ計算機株式会社 | Image processing device, image processing method and program |
CN113556466B (en) * | 2021-06-29 | 2023-01-13 | 荣耀终端有限公司 | Focusing method and electronic equipment |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6671391B1 (en) * | 2000-05-26 | 2003-12-30 | Microsoft Corp. | Pose-adaptive face detection system and process |
-
2006
- 2006-10-11 US US11/545,423 patent/US7693301B2/en active Active
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6671391B1 (en) * | 2000-05-26 | 2003-12-30 | Microsoft Corp. | Pose-adaptive face detection system and process |
Non-Patent Citations (1)
Title |
---|
Demirkir et al. (AVBPA 2005, LNCS 3546 pp. 339-345, 2005, Springer-Verlag, Cem Demirkur and Bulent Dankur, Dept. Electrical-Electronic Engineering Bogazici University) disclose Using Look-Up Table Based Gentle Adaboost. * |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090018985A1 (en) * | 2007-07-13 | 2009-01-15 | Microsoft Corporation | Histogram-based classifiers having variable bin sizes |
US7822696B2 (en) * | 2007-07-13 | 2010-10-26 | Microsoft Corporation | Histogram-based classifiers having variable bin sizes |
US20090116705A1 (en) * | 2007-11-01 | 2009-05-07 | Sony Corporation | Image processing apparatus, image processing method, image processing program, image capturing apparatus, and controlling method thereof |
US8340367B2 (en) * | 2007-11-01 | 2012-12-25 | Sony Corporation | Image processing apparatus, image processing method, image processing program, image capturing apparatus, and controlling method thereof |
US20100008549A1 (en) * | 2008-07-08 | 2010-01-14 | Clay Jessen | Increasing Face Detection Speed |
US8433106B2 (en) * | 2008-07-08 | 2013-04-30 | Hewlett-Packard Development Company, L.P. | Increasing face detection speed |
US20100194903A1 (en) * | 2009-02-03 | 2010-08-05 | Kabushiki Kaisha Toshiba | Mobile electronic device having camera |
US8570431B2 (en) * | 2009-02-03 | 2013-10-29 | Fujitsu Mobile Communications Limited | Mobile electronic device having camera |
US20110293173A1 (en) * | 2010-05-25 | 2011-12-01 | Porikli Fatih M | Object Detection Using Combinations of Relational Features in Images |
US20130089235A1 (en) * | 2011-10-06 | 2013-04-11 | Samsung Electronics Co., Ltd. | Mobile apparatus and method for controlling the same |
US9082016B2 (en) * | 2011-10-06 | 2015-07-14 | Samsung Electronics Co., Ltd. | Mobile apparatus and method for controlling the same |
Also Published As
Publication number | Publication date |
---|---|
US20080089560A1 (en) | 2008-04-17 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7693301B2 (en) | Known face guided imaging method | |
US8306262B2 (en) | Face tracking method for electronic camera device | |
TWI764905B (en) | Apparatus and method for detecting objects, method of manufacturing processor, and method of constructing integrated circuit | |
JP7482181B2 (en) | Image processing device and image processing method | |
US11532154B2 (en) | System and method for providing dominant scene classification by semantic segmentation | |
CN107491762B (en) | A kind of pedestrian detection method | |
US7215828B2 (en) | Method and system for determining image orientation | |
US7929771B2 (en) | Apparatus and method for detecting a face | |
US8761446B1 (en) | Object detection with false positive filtering | |
US6654507B2 (en) | Automatically producing an image of a portion of a photographic image | |
US6545743B1 (en) | Producing an image of a portion of a photographic image onto a receiver using a digital image of the photographic image | |
US7526101B2 (en) | Tracking objects in videos with adaptive classifiers | |
US7920725B2 (en) | Apparatus, method, and program for discriminating subjects | |
US7965886B2 (en) | System and method for detection of multi-view/multi-pose objects | |
US7995807B2 (en) | Automatic trimming method, apparatus and program | |
US8446494B2 (en) | Automatic redeye detection based on redeye and facial metric values | |
US8170332B2 (en) | Automatic red-eye object classification in digital images using a boosting-based framework | |
CN113592911B (en) | Apparent enhanced depth target tracking method | |
Yang et al. | Real-time pedestrian and vehicle detection for autonomous driving | |
US20190012582A1 (en) | Image processing apparatus, image processing method, and non-transitory computer-readable storage medium | |
CN113052170A (en) | Small target license plate recognition method under unconstrained scene | |
US7113637B2 (en) | Apparatus and methods for pattern recognition based on transform aggregation | |
JP4510556B2 (en) | Object identification device and method, and program | |
KhabiriKhatiri et al. | Road Traffic Sign Detection and Recognition using Adaptive Color Segmentation and Deep Learning | |
JP4789526B2 (en) | Image processing apparatus and image processing method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: ARCSOFT, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LI, SHU;WANG, JIN;REEL/FRAME:018409/0266 Effective date: 20060905 Owner name: ARCSOFT, INC.,CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LI, SHU;WANG, JIN;REEL/FRAME:018409/0266 Effective date: 20060905 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
AS | Assignment |
Owner name: EAST WEST BANK,CALIFORNIA Free format text: SECURITY AGREEMENT;ASSIGNOR:ARCSOFT, INC.;REEL/FRAME:024218/0828 Effective date: 20100409 Owner name: EAST WEST BANK, CALIFORNIA Free format text: SECURITY AGREEMENT;ASSIGNOR:ARCSOFT, INC.;REEL/FRAME:024218/0828 Effective date: 20100409 |
|
AS | Assignment |
Owner name: ARCSOFT, INC., CALIFORNIA Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:EAST WEST BANK;REEL/FRAME:026616/0643 Effective date: 20110719 |
|
FEPP | Fee payment procedure |
Free format text: PAT HOLDER NO LONGER CLAIMS SMALL ENTITY STATUS, ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: STOL); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
FPAY | Fee payment |
Year of fee payment: 4 |
|
AS | Assignment |
Owner name: EAST WEST BANK, CALIFORNIA Free format text: SECURITY INTEREST;ASSIGNORS:ARCSOFT, INC.;ARCSOFT (SHANGHAI) TECHNOLOGY CO., LTD.;ARCSOFT (HANGZHOU) MULTIMEDIA TECHNOLOGY CO., LTD.;AND OTHERS;REEL/FRAME:033535/0537 Effective date: 20140807 |
|
AS | Assignment |
Owner name: ARCSOFT HANGZHOU CO., LTD., CALIFORNIA Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:EAST WEST BANK;REEL/FRAME:037109/0027 Effective date: 20151111 Owner name: ARCSOFT, INC., CALIFORNIA Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:EAST WEST BANK;REEL/FRAME:037109/0027 Effective date: 20151111 Owner name: MULTIMEDIA IMAGE SOLUTION LIMITED, CALIFORNIA Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:EAST WEST BANK;REEL/FRAME:037109/0027 Effective date: 20151111 Owner name: ARCSOFT (SHANGHAI) TECHNOLOGY CO., LTD., CALIFORNI Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:EAST WEST BANK;REEL/FRAME:037109/0027 Effective date: 20151111 Owner name: ARCSOFT (HANGZHOU) MULTIMEDIA TECHNOLOGY CO., LTD. Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:EAST WEST BANK;REEL/FRAME:037109/0027 Effective date: 20151111 |
|
FEPP | Fee payment procedure |
Free format text: PAT HOLDER CLAIMS SMALL ENTITY STATUS, ENTITY STATUS SET TO SMALL (ORIGINAL EVENT CODE: LTOS); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2552) Year of fee payment: 8 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 12TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1553); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 12 |