JPH07182518A - Method and device for processing image - Google Patents

Abstract

PURPOSE:To more exactly detect a specified image. CONSTITUTION:ROM 1001-1027 respectively previously store 27 ways of combined three-dimensional color tone distribution information provided by + or - changing (while including the case of no change) three-dimensional color tone distribution information expressed by the RGB space of the same kind of a specified original (such as a paper money of the same amount, for example,) just for a fine amount (m) for each color component. Next, addresses specified by the values of inputted RGB image data are applied to the ROM 1001-1027 and it is investigated whether the color tone of an input image is matched with 27 pieces of color tones stored in the ROM 1001-1027 or not. The result is outputted to OR circuits 1180-1187 and when it is judged that any one of decided results provided from the ROM 1001-1027 is matched with the specified image, it is decided that the color tone of that input image signal is the same as that of the specified image.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method and an apparatus thereof, and more particularly to an image processing method and an apparatus thereof for detecting a specific original based on an input image signal.

[0002]

2. Description of the Related Art In recent years, as the performance of color copying machines and color printers has improved, unauthorized use of these machines has become a problem. That is, there is a problem that these devices are used for forgery of banknotes, securities and the like. Therefore, there is a need for a technique for preventing the duplication of a specific document. As one of the technologies, by pre-registering the distribution information of the tint of a specific document in the color space and comparing it with the tint distribution of the input image data, the input image data can It is proposed in Japanese Patent Application No. 3-160384, etc. to determine whether or not.

Next, the structure of a general color copying machine to which the above technique is applied will be described.

FIG. 6 is a block diagram showing an outline of the entire structure of a color copying machine which is well known in the prior art. In FIG. 6, the mechanism of the scanning unit that mechanically scans the platen on which the image original is placed and the scanner is omitted.

In FIG. 6, 101 is a CCD color sensor, 102 is an analog amplifier, 103 is an A / D converter,
Reference numeral 104 denotes a shading correction circuit for correcting variations in brightness depending on the reading position of the image signal, and 1105 denotes input color signals R (red), G (green), B (blue) for Y (yellow), M (magenta), and C. (cyan),
A print signal generation circuit for converting into a Bk (black) signal. Further, the print signal generation circuit 105 has a built-in delay circuit for compensating the time required for determining the specific image. The print signal output from this circuit is modulated by the real-time correction signal f.

Reference numeral 106 is a color space matching determination circuit for calculating in real time the degree of similarity of the distribution of the read image data and a specific document (for example, bills or securities) in the three-dimensional color space, and 107 is a real-time correction signal. A real-time correction signal generation circuit for generating f, and a read synchronization signal generation circuit for generating a read cycle signal.
This circuit includes a main scanning section signal (HS) 109, a pixel reading basic clock signal (CLK) 110, and a section signal (VS) 11 indicating an effective area in the sub scanning direction for reading an original.
2 is generated. A color printer unit 110 prints out the input image original.

It should be noted that the shading correction circuit 104 corrects the distortion of brightness and tint due to the position of the original, and the shading-corrected color signal output from this circuit is used to change the position of the input original. However, the similarity determination in the color space can be accurately performed, but since this circuit is publicly known, detailed description thereof will be omitted.

FIG. 7 is a block diagram showing the detailed structure of the color space matching determination circuit. In FIG. 7, 1201
Is a signal of the upper 5 bits of the 8-bit R signal output from the shading correction circuit 1104, and 1202 is an 8-bit G signal output from the shading correction circuit 1104.
The upper 5 bit signal of the signal, 1203 is the upper 5 bit signal of the 8-bit B signal output from the shading correction circuit 1104. Reference numeral 1205 is a timing generation circuit that generates various timing signals to be described later, 1209 is an SRAM, and 1270 is an address generator that sequentially generates all addresses of the SRAM 1209.

Reference numeral 1204 denotes a ROM that stores information regarding the tint of a plurality of types of specific originals. When a 5-bit RGB signal is input to the address (A _{0 to} A ₁₄ ) of the ROM, the input RGB signal is output. Judgment signals (X _{0 to} X) that indicate whether or not the colors match a plurality of types of specific originals.
₇ ) is output from the data output terminals (D _{0 to} D ₇ ).

Reference numerals 1220 to 1227 denote smoothing circuits for performing a smoothing operation based on the determination signals (X _{0 to} X ₇ ) output from the ROM 1204. Reference numerals 1240 to 1247 show the similarity between specific image data and an input color signal in RGB space in real time. The color space similarity determination signal (MK _{0 to} MK
_This is a color space determination circuit that outputs ₇ ).

1271-1272 are S according to the address signal generated by the address generator 1270 when the sub-scanning section signal (VS) 112 is "0 (LOW)".
It is a selector used for clearing the RAM 1209 to zero.

FIG. 8 is a diagram conceptually showing the information regarding the tint of a specific document stored in the ROM 1204. As shown in FIG. 8, the tint information of the specific document is expressed by a three-dimensional space whose coordinate axes are RGB color components. In FIG. 8, the shaded area in the three-dimensional space corresponds to the tint of the specific document. Then, whether or not the input RGB signal is a region corresponding to the tint is determined by a combination of RGB coordinate values corresponding to the value of the input RGB signal. Is "1", and if not, the input / output relationship of the ROM 1204 is configured so that the output value is "0". In other words, it can be said that the ROM 1204 stores a table in which the output value is a value of “0” or “1” corresponding to the value of the input RGB signal. The RAM 1 is provided with eight data output terminals (D _{0 to} D ₇ ).
This is because the tint information relating to eight types of specific originals is stored in 204, and each terminal corresponds to one eight types of specific originals.

FIG. 9 is a diagram showing the relationship between the input data and the output data of the data relating to the tint of a plurality of specific originals stored in the ROM 1204. First, of the input addresses (A _{0 to} A ₁₄ ), A _{0 to} A ₄ has a 5-bit B
Signal, G signal 5 bits are A ₅ to A ₉ are, A ₁₀ to A
_A 5-bit R signal is input to ₁₄ . The value of the input address (A _{0 to} A ₁₄ ) is specified by this input signal,
Corresponding to this, the output values for the colors of the eight types of specific originals are determined. Then, for the 5-bit RGB data corresponding to each input pixel, data output terminals (D _{0 to} D
_{From 7} ), the judgment signals (X _{0 to} X ₇ ) relating to the tint of eight different types of specific originals are output in parallel.

FIG. 10 is a circuit diagram showing a circuit configuration of the smoothing circuits 1220 to 1227. In FIG. 10, 160
Reference numerals 1 and 1602 denote multipliers, 1603 denotes an adder, 1604 denotes a latch circuit, and 1605 denotes a comparator. Multiplier 1
The weighted average of the input data and the previous data is taken by 601, 1602 and the adder 1603, and it is possible to make a determination considering continuity as shown in FIG.

FIG. 11 is a diagram showing the relationship between the input value (X _i ) to the smoothing circuit and the smoothed operation value (Y _i ). According to the circuit configuration shown in FIG. 10, when the input value (X _i ) is “1” continuously, the smoothed operation value (Y
_It has the property that the value of _i ) increases. Therefore, when the input RGB signals continuously match the tint of the specific original, the output signals (C _{0 to} C ₇ ) of the smoothing circuit become “1”, which is more accurate without being affected by noise or the like. It is possible to make a judgment.

FIG. 12 shows the color space similarity determination signal (M) by determining the similarity between the specific image data and the input color signal in real time in the RGB space as shown in FIG.
K _{0 to} MK ₇ ) output color space determination circuits 1240-1
It is a block diagram which shows the structure of 247.

The circuit shown in FIG.
Data (D_i : i = 1,8) and smoothing circuit
Output signal (C_i : i = 1,8) is the logical sum (OR) operation
Then, the result is written in the SRAM 1209 again.
In addition, the data (D_i ) Transitions from 0 to 1 only
The counter 1301 is counted up. Counter 1
301 is the rising edge of the sub-scanning section signal (VS) 112.
Cleared. Output value of counter 1301 (Z_i ) And
Constant δ of transistor 1303_i Is the comparator 1302
The size relationship is compared and Z_i > Δ_i In case of, MK_i = 1
Become, Z_i ≤ δ _i In case of, MK_i = 0. δ_i The value of the
U shown in FIG._ORG Value of 1% is set
(Eg l = 90).

Δ _i = (l / 100) × U _ORG where U _ID is each coordinate axis in consideration of the possible values of the 5-bit input RGB data in the RGB coordinate space, as shown in FIG. Is a numerical value in which a unit volume is a cube formed by partitioning into 32 with a predetermined unit length. On the other hand, U
_ORG is the volume of the region formed in the RGB coordinate space by the data representing the tint of the specific image registered in the device.

By the above processing, the observed image data, that is, the data of the input color signal sequence is converted into the specific image data and the RGB image data.
When the shapes are substantially the same in the color space, the color space similarity determination signals (MK _{0 to} MK ₇ ) are “1”.

(3) Control signal time chart (see FIG. 1)
4) FIG. 14 is a diagram showing a timing signal generated by the timing generation circuit 1205. 14, VS is a sub-scanning section signal, CLK is a basic clock, CLK4 is a clock signal obtained by dividing the basic clock (CLK) by four, and OE is applied to the output enable (OE) terminal of the SRAM 1209 through the signal line 1208.
, WE is a signal which is applied to the write enable (WE) terminal of the SRAM 1209 through the signal line 1207 and controls data writing to the SRAM 1209. Also, C _i , C _i ′, D _i ″, EN
B, D _i and D _i ′ respectively show the mutual relation of the control signals or input / output signals shown in FIG.

(4) Configuration of Real-Time Correction Signal Generation Circuit (FIG. 15) FIG. 15 is a block diagram showing the configuration of the real-time correction signal generation circuit 107. As shown in FIG. 15, this circuit is composed of a latch circuit, an OR circuit, and an AND circuit. When it is determined by this circuit that any one of the plurality of specific originals registered in the ROM 1204 matches the input image data in the RGB color space, the real-time correction signal f is “1 (High)”. Become.

(5) Configuration of print signal generation circuit (FIG. 1)
6) FIG. 16 is a block diagram showing the configuration of the print signal generation circuit 105.

In the masking UCR arithmetic circuit 2101,
Normally, each color component YMCB of the density signal rather than the input RGB signal
When the input color signal is determined to match the specific original, the masking UCR calculation circuit 2102 generates the k signal and changes the tint (for example, strengthens the red tint) to generate each color component YMCBk signal of the density signal. To do. Then, the selector 2103 selects and outputs the signal from the masking UCR operation circuit 2101 or 2102 according to the real-time correction signal f.

As a result, it is possible to change the tint and print out only the area where the input image original is determined to match the specific original. Also, when it is determined that the input image original is similar to the specific original, the reproduced image output from the printer is filled with solid black,
Normal image formation may be disabled by turning off the power.

A LOG conversion circuit for converting the RGB signal, which is a 2104 luminance signal, into the YMC signal, which is a density signal.

[0026]

However, in the above-mentioned conventional example, since the input characteristics of the image input section (hereinafter referred to as scanner) of the image processing apparatus are slightly different between apparatuses, even if the same image original is read, The read signal is slightly different for each scanner. Therefore, if the tint distribution information for the specific image determination is created by simply using only the input image data read by one scanner, the determination by the input image data read by the scanner of the other device is not guaranteed. There was a problem.

In addition, among the bills of various countries in the world to be judged as the specific manuscript, there are some bills with large variations in color tone. Therefore, even if the banknotes are of the same type, one banknote can be judged, but another banknote cannot be judged.

Furthermore, a white paper is placed on the back of the specific original so that the copy output image is not greatly affected.
If a copy operation is performed by subtly changing the read signal, there is a problem that the specific document cannot be determined.

Accordingly, in consideration of the above factors, the tint distribution information used for the determination of the specific image has a certain margin, and this margin is necessary in order to avoid misjudgment of the general document as much as possible. To minimize
It is desired to set a delicate balance and set the margin appropriately.

The present invention has been made in view of the above-mentioned conventional example, and more accurately identifies the specific image original in consideration of variations between apparatuses and variations in color tone of the specific image original itself. It is an object of the present invention to provide an image processing method and an apparatus thereof that enable discrimination of a document.

[0031]

In order to achieve the above object, the image processing apparatus of the present invention has the following configuration. That is, it is obtained by adding and subtracting a small amount to each of the first tint information representing the tint of the specific image using a plurality of color components and each of the plurality of color components representing the first tint information. Storage means for storing second tint information, input means for inputting an image original, image data obtained by the input means, the first tint information and the second color stored in the storage means A comparing means for comparing the taste information and a judging means for judging whether or not the image data obtained by the input means represents the specific image based on the comparison result by the comparing means. The image processing device is provided.

According to another aspect of the present invention, the first tint information representing the tint of the specific image using a plurality of color components, and a small amount for each of the plurality of color components representing the first tint information. A storage step of storing in advance a second tint information obtained by adding and subtracting the amount in a storage medium; an input step of inputting an image original; and image data obtained by the input step.
Based on the comparison step of comparing the first tint information and the second tint information stored in the storage medium, and the comparison result of the comparing step, the image data obtained in the input step is identified. And an image processing method for determining whether or not it represents an image.

[0033]

With the above-described structure, according to the present invention, whether the input image data represents the specific image is determined by determining the first tint information representing the tint of the specific image using a plurality of color components, and The second tint information obtained by adding and subtracting a small amount to each of the plurality of color components representing the first tint information is compared with the input image data, and the comparison is performed based on the comparison result.

[0034]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT A preferred embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

FIG. 1 is a block diagram showing the configuration of a color space matching determination circuit incorporated in a color copying machine and used for detecting a specific image, which is a typical embodiment of the present invention. In the constituent elements of the circuit shown in FIG. 1, elements common to those of the conventional circuit shown in FIG. 7 are designated by the same reference numerals, and description thereof will be omitted here. Here, only the characteristic parts of this embodiment will be described. The circuit of the present embodiment is incorporated in a color copying machine having the configuration shown in FIG. 6, and is used. In the description related to the entire copying machine, the components shown in FIG. 6 will be referred to.

In FIG. 1, reference numeral 1201 denotes an upper 5-bit signal of the 8-bit R signal output from the shading correction circuit 1104, and 1202 denotes an upper 5-bit signal of the 8-bit G signal output from the shading correction circuit 1104. Reference numeral 1203 denotes a shading correction circuit 1104.
It is a signal of the upper 5 bits of the 8-bit B signal output from.

Further, reference numerals 1001 to 1027 denote 27 ROs for storing information regarding the tint of a specific image as described later.
It is M. RGB signals 1201 to 1203 are input in parallel to each of these 27 ROMs, and the input RG
The determination signal data indicating whether or not the color tone of the specific original corresponding to the address specified by the B signal matches is output in parallel. With these ROMs, it is possible to store up to 27 conditions (described later) for a specific image of the same type and store information regarding the tint. Since the relationship between the input address and the output data for each ROM is the same as that described in the conventional example, the description thereof will be omitted. Therefore, the color information for eight different specific originals in each of these ROM groups and for each of the 27 different original conditions is stored.

Further, 1180 to 1187 are 27 Rs.
An OR circuit for obtaining the logical sum of the determination signals from the OMs 1001 to 1027, and 2001 to 2027 are 27 ROMs 1
Switches for turning on / off the RGB inputs provided corresponding to each of 001 to 1027, and 3000 for the switch 20.
A setting switch for setting the ON / OFF state of 01 to 2027.

With the above-described structure, the circuit of this embodiment compares the input RGB signals with a maximum of 27 conditions for the same type of specific image at the same time, and any one of them matches the color tone of the specific original. If so, it is determined that the input RGB signal matches the tint of the specific document. If it is not necessary to compare the 27 conditions depending on the operating environment of the apparatus and the nature of the specific original document to be detected, a specific R
The OM may be set so that RGB signals are not input.

Next, 27 ROMs 1001 to 1027
The tint information to be stored in will be described with reference to FIGS. In the following description, only specific images of the same type (for example, banknotes having a certain face value) will be described.

First, a standard specific original (for example, a new bill with a certain face value) is used for the ROM 1001 and is generated without any margin. A three-dimensional tint distribution in the RGB color space corresponding to each pixel data of the input image when the specific document is read is as shown in 2a of FIG. 2, for example. On the other hand, the ROM 1002 has a three-dimensional color distribution 2a.
2, a value “m” that is uniformly added to the value indicated by the B signal causes the distribution of the tint to be 3 as shown by 2b in FIG.
The distribution information, which is translated by + m in the B-axis direction as a whole, is stored without changing the shape of the area indicated by the dimensional color distribution 2a.
Here, the value of m is determined based on the variation of the input sensitivity for each scanner and the degree of the variation of the tint of the specific document to be determined.

Based on the above concept, the ROM 1
In 001 to 1027, each of the three RGB components is independently ± m
Then, the three-dimensional color distribution information of the 27 combinations is stored, including the case where no noise is added.

The three-dimensional color distribution information thus obtained is compared with the input image data. The input image data is rarely obtained from the input image in an ideal state, and noises are superimposed on each of the RGB components more or less. Therefore, as shown in FIGS.
When the three three-dimensional tint distribution information is used, the tint of the specific document may not be detected in some cases. In this embodiment, since 27 types of three-dimensional color distribution information in which noise is taken into consideration for each component are compared with the input image signal, even if a signal on which noise is superimposed is input, a specific original document is input. The color of can be detected. Therefore, according to this embodiment, the specific image can be accurately detected in a wider range in consideration of the condition of the input signal.

In this embodiment, it is assumed that a plurality of tint distributions in the case where a predetermined amount of noise is superimposed on each color component of RGB data representing the tint of a specific document are different from the original tint distribution. Although the distribution and the input RGB signal are compared, the present invention is not limited to this.

For example, when storing the tint distribution information of a specific original in a plurality of ROM groups, as shown in FIG. 5, (1) a white paper is laid on the back of the specific original, and it is optically used as an original C
The image signal is input by scanning with a CD, the first tint distribution information is generated from the input signal, this is stored in the first ROM, and (2) black paper is laid on the back of the specific original, The second tint distribution information obtained in the same manner as in (1) is stored in the second ROM as an original, and (3) the original which is not placed on the back of the specific original is referred to as (1). By storing the third tint distribution information obtained in the same manner in the third ROM, a plurality of different tint distribution information is acquired for the same type of specific original document. Then, using these distribution information, the input RGB
Compare with the signal.

In this way, the tint distribution information that would be generated when a counterfeiting operation is performed by placing a white paper on the back of a specific document can be stored in advance in the ROM. Compared with the three-dimensional shape of the tint distribution of the original specific original, the tint distribution information obtained in this way is shown in FIG. However, the shape is distorted, but its position does not shift significantly because the distribution of dark tints does not change.

In addition to this, a plurality of specific originals are input from a plurality of scanners according to the above-mentioned procedure to acquire tint distribution information, and a new sum like all tint distributions is obtained. Various tint distribution information may be generated and stored in the ROM. This makes it possible to take into account more input characteristics of the scanner, variations in color tint between specific documents of the same type, and the like.

Furthermore, a larger number of tint distribution information may be taken into consideration by combining the case of assuming a predetermined amount of noise superimposition described in the present embodiment and the above-mentioned difference in input conditions.

Note that there is a risk that the tint distribution detected as the tint of the specific original will be unnecessarily widened if the tint distribution information under a number of different conditions regarding the same type of specific original is taken into consideration. The circuit may be configured to analyze the tendency of the information distribution and obtain the logical product of the determination signal data output from the specific ROM. For example, in the above example, the logical product of at least two of the comparison results is taken, not the simple logical sum of the comparison results of the first, second, and third color distribution information and the input signal. The circuit is configured as follows.

Furthermore, before the information is stored in the ROM, the tint distribution information is analyzed to identify the tint distribution information which is not affected by scanner input characteristics, tint variation of a particular original, artificial data disturbance due to forgery, etc. It is also possible to obtain more essential color distribution information of the manuscript.

The present invention may be applied to a system composed of a plurality of devices or an apparatus composed of one device. Further, it goes without saying that the present invention can be applied to the case where it is achieved by supplying a program to a system or an apparatus.

[0052]

As described above, according to the present invention, it is possible to determine whether the input image data represents the specific image by using the first color component representing the tint of the specific image.
The tint information and the second tint information obtained by adding and subtracting a minute amount to each of the plurality of color components representing the first tint information are compared with the input image data, and the comparison result is obtained. Since it is performed based on the above, the comparison between the specific image and the input image can be performed in a more multifaceted manner, and there is an effect that it is possible to more accurately determine the specific image. As a result, it is possible to comprehensively determine the specific image including the reading variation caused by the variation of the input device characteristics for inputting the image.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a color space matching determination circuit that is a typical embodiment of the present invention.

FIG. 2 is a diagram illustrating a change in tint distribution when noise (m) is added to a B component signal of an input image.

FIG. 3 is a diagram for explaining how to determine the tint of an input image to which noise has been added.

FIG. 4 is a diagram for explaining how to determine the tint of an input image to which noise has been added.

FIG. 5 is a diagram illustrating a change in tint distribution when a white / black paper of a specific original is laid.

FIG. 6 is a block diagram showing an outline of the entire configuration of a generally known color copying machine.

FIG. 7 is a block diagram showing a detailed configuration of a color space matching determination circuit.

8 is a diagram conceptually showing information relating to the tint of a specific document stored in a ROM 1204 shown in FIG.

9 is a diagram showing a relationship between input data and output data with respect to data relating to the tint of a plurality of specific originals stored in a ROM 1204 shown in FIG.

10 is a circuit diagram showing a circuit configuration of smoothing circuits 1220 to 1227 shown in FIG.

FIG. 11 is a diagram showing a relationship between an input value (X _i ) to a smoothing circuit and a smoothed calculation value (Y _i ).

FIG. 12 is a block diagram showing a configuration of color space determination circuits 1240 to 1247 that calculate color space similarity determination signals (MK _{0 to} MK ₇ ).

FIG. 13 is a diagram showing the tint characteristics of the specific image and the input image expressed in the RGB coordinate space.

FIG. 14 is a diagram showing a timing signal generated by a timing generation circuit 1205.

FIG. 15 is a block diagram showing the configuration of a real-time correction signal generation circuit 1107.

FIG. 16 is a block diagram showing a configuration of a print signal generation circuit 1105.

[Explanation of symbols]

 1001-1027 ROM 1180-1187 OR circuit 2001-2027 switch 3000 setting switch

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location H04N 1/40 1/46 G06F 15/62 410 Z H04N 1/40 Z 1/46 Z (72) Inventor Go Aoyagi 3-30-2 Shimomaruko, Ota-ku, Tokyo Canon Inc.

Claims (7)

[Claims] 1. A small amount is added to or subtracted from each of the first tint information representing the tint of a specific image using a plurality of color components and each of the plurality of color components representing the first tint information. Storage means for storing the second tint information obtained by the above, input means for inputting an image original, image data obtained by the input means, the first tint information stored in the storage means, and the above-mentioned Comparing means for comparing the second tint information, and judging means for judging whether or not the image data obtained by the input means represents the specific image based on the comparison result by the comparing means. An image processing apparatus having. 2. The image processing apparatus according to claim 1, wherein the storage unit has a plurality of ROMs that store the first tint information and the second tint information, respectively. 3. The image processing apparatus according to claim 1, wherein the specific image includes an image representing banknotes and securities. 4. The input means is an optical scanner, and the minute amount is a margin obtained from a variation in input sensitivity of the optical scanner and a variation in tint peculiar to the specific image. Item 1. The image processing device according to item 1. 5. The determination means includes a first comparison result of the image data obtained by the input means and the first tint information, and a second comparison result of the image data obtained by the input means. The image processing apparatus according to claim 1, wherein the logical sum of is used as a comparison result by the comparison means. 6. The determination means includes a first comparison result of the image data obtained by the input means and the first tint information, and a second comparison result of the image data obtained by the input means. The image processing apparatus according to claim 1, wherein a logical product of the two is used as a comparison result by the comparison means. 7. A first amount of tint information representing the tint of a specific image using a plurality of color components, and a small amount is added to or subtracted from each of the plurality of color components representing the first tint information. A storage step of preliminarily storing the second tint information obtained by the above in a storage medium, an input step of inputting an image original, image data obtained in the input step, and the first color stored in the storage medium. Based on the comparison step of comparing the taste information and the second tint information, and based on the comparison result of the comparison step, it is determined whether or not the image data obtained by the input step represents the specific image. An image processing method comprising: a determining step.