JPH077440B2 - Image processing device - Google Patents

Image processing device

Info

Publication number
JPH077440B2
JPH077440B2 JP61071345A JP7134586A JPH077440B2 JP H077440 B2 JPH077440 B2 JP H077440B2 JP 61071345 A JP61071345 A JP 61071345A JP 7134586 A JP7134586 A JP 7134586A JP H077440 B2 JPH077440 B2 JP H077440B2
Authority
JP
Japan
Prior art keywords
color
image
image processing
processing apparatus
skin color
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
JP61071345A
Other languages
Japanese (ja)
Other versions
JPS62229478A (en
Inventor
英夫 滝口
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Canon Inc
Original Assignee
Canon Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Canon Inc filed Critical Canon Inc
Priority to JP61071345A priority Critical patent/JPH077440B2/en
Publication of JPS62229478A publication Critical patent/JPS62229478A/en
Priority to US07/408,424 priority patent/US5130935A/en
Publication of JPH077440B2 publication Critical patent/JPH077440B2/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Landscapes

  • Processing Or Creating Images (AREA)
  • Image Processing (AREA)
  • Facsimile Image Signal Circuits (AREA)
  • Color Image Communication Systems (AREA)

Description

【発明の詳細な説明】 [産業上の利用分野] 本発明は画像処理装置、特に色補正を行なう画像処理装
置に関するものである。
The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus that performs color correction.

[従来の技術] 従来、画像の色補正は印刷の分野等で行なわれ、例えば
人の顔の肌色が緑がかつているときには好ましい肌色へ
と補正を行なつている。しかしながら、これらの補正の
判断、またその補正量は殆ど人の経験と勘で決定され、
行なわれているのが現状である。そこで、原画中の色の
中でも人が最も注目する色である肌色に注目し、ある一
定値以上肌色が存在する場合には、補正量を求め、自動
的に画像全体の色調を補正することにより、肌色重視の
色補正を実現する。
[Prior Art] Conventionally, image color correction has been performed in the field of printing and the like, for example, when the skin color of a human face is green, it is corrected to a preferable skin color. However, the judgment of these corrections, and the amount of the corrections are determined mostly by human experience and intuition,
This is the current situation. Therefore, among the colors in the original picture, pay attention to the skin color, which is the color most noticed by humans, and if there is a skin color above a certain value, calculate the correction amount and automatically correct the tone of the entire image. Realizes color correction that emphasizes skin color.

[発明が解決しようとする問題点] 本発明は画像の色補正の必要性を判断し、色補正量を適
正化した画像処理装置を提供する。
[Problems to be Solved by the Invention] The present invention provides an image processing apparatus that determines the necessity of color correction of an image and optimizes the amount of color correction.

[問題点を解決するための手段] この問題点を解決する一手段として、第1図に示す画像
処理装置は、Luv変換部1と、肌色カウント1部2と、
肌色カウント2部3と、画像数判定部4と、最大値判定
部5と、補正量演算部6と、色補正部7と、RGB変換部
8と、信号線10とを備える。又第4図に示す画像処理装
置は、CPU40と、RAM41と、ROM42と、画像データ入力部4
3と、色補正データ出力部とを備える。
[Means for Solving Problems] As one means for solving this problem, the image processing apparatus shown in FIG. 1 includes a Luv conversion unit 1, a skin color counting unit 1
The skin color counting unit 2 includes an image number determining unit 4, a maximum value determining unit 5, a correction amount calculating unit 6, a color correcting unit 7, an RGB converting unit 8, and a signal line 10. The image processing apparatus shown in FIG. 4 includes a CPU 40, a RAM 41, a ROM 42, and an image data input unit 4
3 and a color correction data output unit.

[作用] かかる構成において、まず図示しない画像入力装置によ
りRGB値を得る。RGB値をLuv変換部1でL*u′v′値
に変換する。そして肌色カウント1部2で、u′v′色
度図上で定めた肌色領域に落ちる画素数をカウントす
る。次に肌色カウント2部3で肌色領域を碁盤の目に分
割し、肌色領域の中のどこに落ちたのかをカウントして
おく。次に、画素数判定部4で肌色領域に落ちた画素数
と、しきい値αとを比較し、画素数<αのときは画像中
に肌色はないか又は少ないとみなし、処理を終了する。
画素数≧αのときは最大値判定部5で、肌色領域のどこ
に一番多く落ちたのかを求める。補正量演算部6で基準
肌色との色差を求め、補正量を算出する。これを信号線
10を通して色補正部7に送信する。次に画像の色補正を
行なう。図示しない画像入力装置により、RGB値を得Luv
変換部1でL*u′v′に変換し、色補正部7でL*は
そのままで、u′←u′−△u,v′←v′−△vとして
補正を行ないRGB変換部8でRGB値に戻し、色補正された
画像を得て処理を終了する。
[Operation] In such a configuration, first, RGB values are obtained by an image input device (not shown). The RGB value is converted into an L * u′v ′ value by the Luv converter 1. Then, the skin color counting unit 1 counts the number of pixels falling in the skin color area defined on the u′v ′ chromaticity diagram. Next, the skin color area is divided into squares on the board by the skin color counting section 2 and the place where the skin color area falls is counted in advance. Next, the number-of-pixels determination unit 4 compares the number of pixels that have fallen in the flesh-colored area with a threshold value α, and when the number of pixels <α, considers that the image has no or little flesh-color and ends the process. .
When the number of pixels ≧ α, the maximum value determination unit 5 finds where in the flesh-colored area the most drops have occurred. The correction amount calculation unit 6 obtains the color difference from the reference skin color and calculates the correction amount. This is the signal line
It transmits to the color correction part 7 through 10. Next, color correction of the image is performed. Get RGB values with an image input device (not shown) Luv
The conversion unit 1 converts to L * u′v ′, and the color correction unit 7 corrects L * as it is, u ′ ← u′−Δu, v ′ ← v′−Δv, and the RGB conversion unit 8 Return to RGB values with to obtain a color-corrected image and end the process.

一方、第4図は肌色カウント1部2から補正量演算部6
までをコンピユータに置き換えたもので、画像データ入
力部43から入力したL*u′v′成分から、CPU40でRAM
42を使用しながらROM1に格納されたプログラムに従つて
補正量を算出し色補正データ出力部44から信号線10を通
して色補正部7に出力する。
On the other hand, FIG. 4 shows the flesh color count part 1 to the correction amount calculation part 6
The above is replaced with a computer. From the L * u'v 'component input from the image data input unit 43, the CPU 40 RAM
While using 42, the correction amount is calculated according to the program stored in the ROM 1 and is output from the color correction data output unit 44 to the color correction unit 7 through the signal line 10.

[実施例] 以下、本発明の実施例を図面を参照しつつ、詳細に説明
する。第1図は実施例の画像処理装置のブロツク図であ
り、画像はレツド,グリーン,ブルー(以下RGB)値で
構成されているものとする。第1図の1〜6は画像中よ
り肌色を抽出して補正量を求める部分、7〜8は補正量
に従つて画像の色補正をする部分である。初めに肌色を
抽出して、補正量を求める処理から説明していく。
Embodiments Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 1 is a block diagram of the image processing apparatus of the embodiment, and it is assumed that the image is composed of red, green, and blue (hereinafter RGB) values. 1 to 6 in FIG. 1 are portions for extracting a skin color from the image to obtain a correction amount, and 7 to 8 are portions for performing color correction of the image according to the correction amount. First, the process of extracting the skin color and obtaining the correction amount will be described.

まず原画より図示しない画像入力装置によりRGB値を得
る。このとき全画素のRGB値は必要ではなく、画像の端
の部分を除いた中央の部分で、その中でも1〜2画素お
きで十分である。理由は人の顔など肌色は画像の中央に
位置していることが多いためである。こうして得たRGB
値をLuv変換部1でL*u′v′値に変換する。そして
肌色カウント1部2で、第2図に示すu′v′色度図上
で定めた肌色領域に落ちる画素数をカウントする。本例
の肌色領域は、 (u′,v′)が(0.22,0.46) (0.26,0.46) (0.32,0.50) (0.32,0.54) (0.28,0.54) (0.22,0.50) で囲まれ、30≦L*≦80の領域とした。しかし、これは
決定的なものではなく、値はこれと違つても構わない。
次に肌色カウント2部3で肌色領域を第3図に示すよう
な碁盤の目に分割し、肌色領域の中のどこに落ちたのか
をカウントしておく。第3図は本例では、肌色領域を
u′,v′方向とも0.05おきに分割し、配列A[16,20]
とした。次に画素数判定部4で肌色領域に落ちた画素数
と、しきい値αとを比較し、画素数<αのときは画像中
に肌色はないか又は少ないとみなし、処理を終了する。
しきい値αは調べた画素数の1/3〜1/4程度が適当であ
る。画素数≧αのときは最大値判定部5で、肌色領域の
どこに一番多く落ちたのかを配列Aの最大値を得ること
により求め、この時のu′,v′の値を〈max〉u,〈max〉
vで表わす。補正量演算部6で基準肌色と、〈max〉u,
〈max〉vとの色差を求める。基準肌色は第2図に示す
直線で本例では、(0.20,0.46)と(0.32,0.54)を結ぶ
直線とした。色差△u、△vは〈max〉u,〈max〉vを通
る直線と、基準肌色直線が直交する点をsu,svとすると
きに△u=maxu−su,△v=〈max〉v−svで求める。s
u,svは以下の式で求める。
First, RGB values are obtained from the original image by an image input device (not shown). At this time, the RGB values of all the pixels are not necessary, and the central part excluding the edge part of the image, and every 1 to 2 pixels among them, is sufficient. The reason is that skin colors such as human faces are often located in the center of the image. RGB thus obtained
The value is converted into an L * u′v ′ value by the Luv converter 1. Then, the skin color count unit 1 counts the number of pixels falling in the skin color area defined on the u'v 'chromaticity diagram shown in FIG. In the skin color area of this example, (u ′, v ′) is surrounded by (0.22,0.46) (0.26,0.46) (0.32,0.50) (0.32,0.54) (0.28,0.54) (0.22,0.50), and 30 The region was ≤L * ≤80. However, this is not critical and the value can be different.
Next, the flesh color count 2 section 3 divides the flesh color region into the grid patterns shown in FIG. 3 and counts where the flesh color region falls. FIG. 3 shows that in this example, the flesh color area is divided every 0.05 in both the u ′ and v ′ directions, and the array A [16,20] is used.
And Next, the pixel number determination unit 4 compares the number of pixels that have fallen in the flesh-colored area with a threshold value α, and when the number of pixels <α, it is considered that there is no or little flesh-colored image in the image, and the process ends.
It is suitable that the threshold value α is about 1/3 to 1/4 of the number of examined pixels. When the number of pixels ≧ α, the maximum value determination unit 5 finds where in the flesh color area the most drops occur by obtaining the maximum value of the array A, and the values of u ′ and v ′ at this time are <max>. u, 〈max〉
Represented by v. In the correction amount calculation unit 6, the reference skin color and <max> u,
Find the color difference from <max> v. The reference skin color is the straight line shown in FIG. 2, and in this example, the straight line connecting (0.20, 0.46) and (0.32, 0.54). Color differences Δu and Δv are Δu = maxu-su, Δv = <max> v, where su and sv are points at which the straight line passing through <max> u and <max> v and the reference skin color straight line are orthogonal. Calculate with −sv. s
u and sv are calculated by the following formula.

基準肌色直線がv=au+b1のとき、〈max〉u,〈max〉v
を通り直交する直線は、v=1/au+b2で表わせる。但
し、b2=(〈max〉v−1/a〈max〉u)である。この
時、 su=(b2−b1)/(a−1/a) sv=a・su+b1である。
When the standard skin color line is v = au + b 1 , <max> u, <max> v
A straight line passing through and can be expressed by v = 1 / au + b 2 . However, b 2 = (<max> v−1 / a <max> u). At this time, su = (b 2 −b 1 ) / (a−1 / a) sv = a · su + b 1 .

これで補正量△u,△vが求まつたので、これを信号線10
を通して色補正部7に送信する。次に画像全体の色補正
を行なう。原画から図示しない画像入力装置により、RG
B値を得Luv変換部1でL*u′v′に変換し、色補正部
7でL*はそのままで、u′←u′−△u、v′←v′
−△vとして補正を行ないRGB変換部8でRGB値に戻し、
色補正された画像を得て処理を終了する。
With this, the correction amounts Δu and Δv have been obtained.
To the color correction unit 7. Next, color correction of the entire image is performed. RG from the original image by an image input device (not shown)
The B value is obtained and converted into L * u′v ′ by the Luv conversion unit 1 and u * ← u′−Δu, v ′ ← v ′ while L * remains as it is in the color correction unit 7.
-Correction is performed as Δv and the RGB value is returned to the RGB value by the RGB conversion unit 8,
The color-corrected image is obtained, and the process ends.

第4図に他の実施例の画像処理装置のブロツク図を示
す。これは第1図の肌色カウント1部2から補正量演算
部6までをコンピユータに置き換えたもので、画像デー
タ入力部43から入力したL*u′v′成分から、CPU40
でRAM42を使用しながらROM1に格納されたプログラムに
従つて補正量を算出し色補正データ出力部44から信号線
10を通して色補正部7に出力する。
FIG. 4 shows a block diagram of an image processing apparatus of another embodiment. This is a computer in which the flesh color count part 1 to the correction amount calculation part 6 in FIG. 1 are replaced with a computer. From the L * u′v ′ component input from the image data input part 43, the CPU 40
While using RAM 42, calculate the correction amount according to the program stored in ROM 1 and output the signal from color correction data output unit 44.
Output to the color correction unit 7 through 10.

第5図の補正量算出のフローチヤートに基づいてその動
作を説明する。
The operation will be described based on the flow chart for calculating the correction amount in FIG.

まずステツプS51でRAM42等の初期化を行う。ステツプS5
2で画像データ入力部43より1画素のL*u′v′成分
を受け取る。ステツプS53で受け取つた画素が肌色領域
に入つているかをチエツクし、肌色でなければステツプ
S56に飛ぶ。肌色の場合は、ステツプS54で肌色の画素数
をカウントしているカウンタを1つカウントアツプし、
ステツプS55で第3図に示したu′v′色度図上のA[1
6:20]内の位置によりその出現数をカウントアツプす
る。ステツプS56では全ての画素をチエツクしたか判断
し、まだ有る場合はステツプS52に戻つて、ステツプS52
〜56を繰り返す。尚、チエツクする画素は全画面分を必
要としないのは前述の通りである。ステツプS57では肌
色の出現確率を計算し、ステツプS58で出現確率がしき
い値αより小さい場合には肌色が少ないと判断して補正
をせずに終了する。一方、肌色の出現確率がしきい値α
より大きい場合はステツプS59でステツプS55によつて作
成されていたu′v′色度図上の出現回数の最大の色を
見付け、ステツプS60でその色の基準色からの偏りの補
正値を算出してステツプS61で色補正データ出力部44か
ら信号線10を通して色補正部7に出力する。補正値の算
出法は前述したものと同じである。
First, in step S51, the RAM 42 and the like are initialized. Step S5
At 2, the L * u'v 'component of one pixel is received from the image data input unit 43. Check whether the pixel received in step S53 is in the skin color area. If it is not skin color, step
Fly to S56. In the case of flesh color, in step S54, count up one counter that counts the number of flesh color pixels,
At step S55, A [1 on the u'v 'chromaticity diagram shown in FIG.
6:20] and count up the number of appearances. In step S56, it is judged whether all the pixels have been checked, and if there are still pixels, the process returns to step S52, and step S52
Repeat ~ 56. As described above, the pixels to be checked do not need to cover the entire screen. In step S57, the appearance probability of the skin color is calculated. If the appearance probability is smaller than the threshold value α in step S58, it is determined that the skin color is small, and the process ends without correction. On the other hand, the appearance probability of the skin color is the threshold α
If it is larger, in step S59, the color having the largest number of appearances in the u'v 'chromaticity diagram created by step S55 is found, and in step S60, the correction value of the deviation from the reference color is calculated. Then, in step S61, the data is output from the color correction data output unit 44 to the color correction unit 7 through the signal line 10. The method of calculating the correction value is the same as that described above.

尚、本実施例においては、RGBからL*u′v′へ変換
して処理を行なつたが、L*u′v′の他の演算色空間
座標系でも実施可能である。又、入力画像データもRGB
であつてもYMCであつても他の伝送入力されたデータで
あつても良い。
In this embodiment, RGB is converted into L * u'v 'for processing, but it is also possible to use another calculation color space coordinate system of L * u'v'. The input image data is also RGB
, YMC, or other transmitted and input data.

以上の様に本実施例によれば、画像中の肌色の最も集ま
つている点を好ましい色相の肌色へとシフトすることに
より、自動的に画像中の肌色を好ましい肌色へと色修正
することが可能となる。
As described above, according to the present embodiment, the skin color in the image is automatically corrected to the preferred skin color by shifting the most concentrated points of the skin color in the image to the skin color of the preferred hue. Is possible.

[発明の効果] 本発明によれば、画像の色補正の必要性を判断し、色補
正量を適正化する画像処理装置を提供できる。
EFFECTS OF THE INVENTION According to the present invention, it is possible to provide an image processing apparatus that determines the necessity of color correction of an image and optimizes the color correction amount.

【図面の簡単な説明】[Brief description of drawings]

第1図は実施例の画像処理装置のブロツク図、 第2図はu′v′色度図上で定めた肌色の領域と基準肌
色の直線を示す図、 第3図は第2図で定めた肌色領域の分割例を示す図、 第4図は他の実施例の画像処理装置のブロツク図、 第5図は第4図の画像処理装置の動作フローチヤートで
ある。 図中、1……Luv変換部、2……肌色カウント1部、3
……肌色カウント2部、4……画像数判定部、5……最
大値判定部、6……補正量演算部、7……色補正部、8
……RGB変換部、10……信号線、40……CPU、41……RA
M、42……ROM、43……画像データ入力部、44……色補正
データ出力部である。
FIG. 1 is a block diagram of the image processing apparatus of the embodiment, FIG. 2 is a diagram showing a skin color region defined on the u′v ′ chromaticity diagram and a straight line of the reference skin color, and FIG. 3 is defined in FIG. 4 is a block diagram of an image processing apparatus according to another embodiment, and FIG. 5 is an operation flow chart of the image processing apparatus of FIG. In the figure, 1 ... Luv conversion part, 2 ... skin color count part 1, 3
...... Skin color counting section 2, 4 ... Image number determining section, 5 ... Maximum value determining section, 6 ... Correction amount calculating section, 7 ... Color correcting section, 8
...... RGB converter, 10 ...... Signal line, 40 ...... CPU, 41 ...... RA
M, 42 ... ROM, 43 ... Image data input section, 44 ... Color correction data output section.

Claims (5)

【特許請求の範囲】[Claims] 【請求項1】画像データを処理して出力する画像処理装
置において、画像に存在する所定の色の量を測定する測
定手段と、該測定手段の測定結果から前記所定の色が所
定量以上前記画像に存在する場合に、前記色の基準色か
らの偏りに基づいて前記画像を色補正する色補正手段と
を備えることを特徴とする画像処理装置。
1. An image processing apparatus for processing and outputting image data, wherein a measuring means for measuring an amount of a predetermined color existing in an image, and a predetermined amount or more of the predetermined color based on a measurement result of the measuring means. An image processing apparatus, comprising: a color correcting unit that corrects the color of the image based on a deviation of the color from a reference color when the color exists in the image.
【請求項2】測定手段は基準色を含む所定範囲内の色を
所定の色と判断することを特徴とする特許請求の範囲第
1項記載の画像処理装置。
2. The image processing apparatus according to claim 1, wherein the measuring means determines a color within a predetermined range including the reference color as a predetermined color.
【請求項3】基準色よりの偏りは最も多く出現した色の
基準色よりの偏りであることを特徴とする特許請求の範
囲第2項記載の画像処理装置。
3. The image processing apparatus according to claim 2, wherein the deviation from the reference color is the deviation of the most frequently appearing color from the reference color.
【請求項4】所定の色は肌色であることを特徴とする特
許請求の範囲第1項記載の画像処理装置。
4. The image processing apparatus according to claim 1, wherein the predetermined color is a skin color.
【請求項5】画像データはL*u′v′成分を基に処理
されることを特徴とする特許請求の範囲第1項記載の画
像処理装置。
5. The image processing apparatus according to claim 1, wherein the image data is processed based on the L * u'v 'component.
JP61071345A 1986-03-31 1986-03-31 Image processing device Expired - Fee Related JPH077440B2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP61071345A JPH077440B2 (en) 1986-03-31 1986-03-31 Image processing device
US07/408,424 US5130935A (en) 1986-03-31 1989-09-14 Color image processing apparatus for extracting image data having predetermined color information from among inputted image data and for correcting inputted image data in response to the extracted image data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP61071345A JPH077440B2 (en) 1986-03-31 1986-03-31 Image processing device

Publications (2)

Publication Number Publication Date
JPS62229478A JPS62229478A (en) 1987-10-08
JPH077440B2 true JPH077440B2 (en) 1995-01-30

Family

ID=13457813

Family Applications (1)

Application Number Title Priority Date Filing Date
JP61071345A Expired - Fee Related JPH077440B2 (en) 1986-03-31 1986-03-31 Image processing device

Country Status (1)

Country Link
JP (1) JPH077440B2 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100237637B1 (en) * 1996-06-04 2000-01-15 구자홍 Skin color reproduction apparatus and method
JP2005328845A (en) * 2004-05-06 2005-12-02 Oce Technologies Bv Methods, apparatus and computer for transforming digital colour images
JP4978516B2 (en) 2008-03-06 2012-07-18 ブラザー工業株式会社 Print data creation apparatus, print data creation program, and computer-readable recording medium
JP4998352B2 (en) * 2008-03-31 2012-08-15 ブラザー工業株式会社 Print data creation apparatus, print data creation program, and computer-readable recording medium

Also Published As

Publication number Publication date
JPS62229478A (en) 1987-10-08

Similar Documents

Publication Publication Date Title
US5130935A (en) Color image processing apparatus for extracting image data having predetermined color information from among inputted image data and for correcting inputted image data in response to the extracted image data
US7683973B2 (en) Image processing method and apparatus performing color adjustment
US6614471B1 (en) Luminance correction for color scanning using a measured and derived luminance value
US6229580B1 (en) Image color correction apparatus and recording medium having color program recorded thereon
US6947078B1 (en) Color correction apparatus, color correction method, and recording medium having color correction control program recorded
US20090034838A1 (en) Device adaptively switching color emphasis processing for image
US6741753B1 (en) Method and system of local color correction using background liminance masking
JP2002101422A (en) Image processing unit, image processing method and computer-readable recording medium for recording image processing program
US6744920B1 (en) Method, apparatus, and recording medium for processing image data to obtain color-balance adjusted image data based on white-balance adjusted image data
US7289665B2 (en) Image processing device and image processing program
US20070013929A1 (en) Method of converting image signal, method of determining original color of color image, and apparatus for converting image signal
JP2001008220A (en) Color reproduction system
US7340104B2 (en) Correction parameter determining method, correction parameter determining apparatus, computer program, and recording medium
JP2001268391A (en) Luminance gradation correcting device for video signal
JP2000050318A (en) Method for characterizing response function of output device, program product and system
JPH0789368B2 (en) Image processing device
US8873108B2 (en) Image processing apparatus and image processing method
JPH077440B2 (en) Image processing device
EP0842496B1 (en) Method and apparatus for maximizing the visual quality of image presented in electric form
JP2000135819A (en) Method for determining gray value in printer
US7319544B2 (en) Processing of digital images
JP4057662B2 (en) Image processing apparatus and image processing method
JP2648206B2 (en) Method of setting output value correction table in image input / output device
JPH09284429A (en) Picture evaluation device
JP2002247396A (en) Method and device for correcting color of printer

Legal Events

Date Code Title Description
LAPS Cancellation because of no payment of annual fees