JP5401645B2 - Human interface device - Google Patents

Description

  The present invention relates to a human interface device for a device such as a computer, and more particularly to a human interface device that outputs three-dimensional position information of a human finger.

  Since the concept of GUI (Graphical User Interface) was invented in the 1970s, various pointing devices such as a mouse, a trackball, a joystick, and a pen input tablet have been used for inputting information to a computer. In particular, the spread of mice is remarkable, and until now, no pointing device that exceeds the mouse has been known in terms of ease of use and cost.

  Although the current pointing device seems to input the displacement of the two-dimensional coordinate continuously, the data is repeatedly input at a fixed period, and only one point coordinate can be input at a certain moment. . That is, it is not possible to input multiple points of displacement and coordinates simultaneously. In addition, the displacement and coordinates are on a two-dimensional plane, and it is generally not possible to input three-dimensional coordinates obtained by adding a coordinate in the height direction. As a three-dimensional coordinate input device developed for CAD and games, a “3d mouse” shown in FIG. 10 is known (see Non-Patent Document 1 below). This is to detect displacement in each direction using a plurality of sensors.

  On the other hand, Patent Document 1 below discloses a human interface device having a function of detecting the positions of a plurality of fingers. FIG. 11 shows a cross-sectional configuration of this apparatus. The light output from the illumination device is reflected by the outer shell pattern, and an image of the outer shell pattern is acquired by the imaging optical system and the imaging device. When this device is supported by a hand with a plurality of fingers, information on the position of the finger in contact with the outer surface of the outer shell can be obtained. Further, by configuring the outer shell with a flexible material, information on the pressure applied to the outer shell by the finger can be obtained based on the image of the deformed outer shell pattern.

Japanese Patent No. 4169688

Internet <URL: http://www.nissho-ele.co.jp/product/3d_mouse/feature.html>

  As described above, there is a demand for a user interface device that can simultaneously input displacements and coordinates of a plurality of points. If the displacements and coordinates of a plurality of points can be input simultaneously, a natural interface can be realized in which images of pointers and fingers corresponding to these points are displayed on the display and the objects displayed at the same time are operated. That is, an interface using finger movement (gesture) can be realized by simultaneously inputting a plurality of finger displacements and coordinates.

  Also, recently, research and development of 3d display has been active. In the future, a large amount of 3d content such as movies and games will be developed, and it is expected that home television and personal computer monitors will have a three-dimensional display function. In such an environment, a user interface for inputting three-dimensional coordinates is required to operate a three-dimensionally displayed virtual object. Moreover, if multi-point coordinates can be input at the same time, the computer can be operated by the movement of a plurality of fingers. For example, the virtual object displayed on the 3d display can be operated as if it were actually picked up.

  However, the existing pointing device is remarkably limited in function as a computer user interface. For example, the input operation using the 3d mouse in Non-Patent Document 1 cannot be said to be a natural input operation for humans, and has not yet replaced a general 2d mouse. Further, since the device of Patent Document 1 is based on the premise that the finger is in contact with the device, no information can be input when the finger is separated from the device.

  Therefore, an object of the present invention is to provide a human interface device capable of inputting information by a natural operation of a person.

  The object of the present invention is achieved by the following means.

That is, the human interface device according to the present invention includes light emitting means, imaging means, and image processing means,
In a state where the light emitting means radiates light to the object, the imaging means detects light reflected from the object and acquires an image,
The image processing means
In the image, an image area corresponding to a specific part of the object surface is detected,
Using the detected feature amount of the image area and the conversion data, obtain a distance from a reference position to the specific portion of the object surface,
The conversion data is data that associates the distance from the reference position with the feature amount.

The feature quantity is the size of the image area.

The object may be a human hand, and the specific portion may be a tip portion of a finger.

  Further, the light is near-infrared light, the imaging unit includes a fisheye lens, and the image processing unit determines the position of the specific part in polar coordinates from the position of the specific part on the image and the optical characteristics of the imaging unit. The polar angle and the azimuth angle can be obtained, and the distance, the polar angle, and the azimuth angle can be determined as the three-dimensional position information of the specific portion.

  Further, the image processing means corrects the conversion data relating to brightness using the polar angle dependency of the light distribution characteristic of the light emitted from the light emitting means and the polar angle of the specific portion. Further, the distance can be obtained using the conversion data.

  According to the present invention, it is possible to simultaneously acquire three-dimensional position information related to a specific portion of an object, in particular, three-dimensional position information of the tip of each finger of a person. Accordingly, it becomes possible to simultaneously input displacements and coordinates of a plurality of points to a computer or the like.

  Therefore, it is possible to realize a natural interface in which images of pointers and fingers corresponding to these are displayed on the display and the objects displayed at the same time are operated. That is, an interface using finger movement (gesture) can be realized by simultaneously inputting a plurality of finger displacements and coordinates.

  In the future, if a home TV or PC monitor has a three-dimensional display function and a large amount of 3d contents such as movies and games will be developed, the present invention can be applied to move multiple fingers. Thus, the virtual object displayed on the 3d display can be manipulated as if it were actually picked up. For example, when searching for files stored in a computer storage device, multiple fingers and cabinets are displayed three-dimensionally on the display, a labeled drawer is opened, and multiple files stored in folders inside Use your finger to find the desired product while turning it with your fingers, or to display the product and five fingers in 3D and move the product with your fingers to observe the product from various angles and check its functions. Is possible.

It is a block diagram showing a schematic structure of a human interface device concerning an embodiment of the invention. It is a figure which shows an example of the light emission means and imaging means of the human interface apparatus shown in FIG. It is a figure which shows a polar coordinate. It is a graph which shows the light distribution characteristic of the light emission means made as an experiment. It is the image imaged using the light emission means made as an experiment. It is a graph of the average value of the width of the finger and the pixel value at the center of the finger obtained from the image captured using the prototype light emitting means. It is a graph which shows the light distribution characteristic of another light-emitting means made as an experiment. It is a graph of the average value of the pixel width | variety of the finger | toe width | variety obtained from the image imaged using another light emission means made as an experiment, and the finger | toe center part. It is a flowchart which shows the operation | movement which acquires the positional infomation on a fingertip using the human interface apparatus which concerns on embodiment of this invention. It is a perspective view which shows an example of the conventional 3d mouse | mouth. It is sectional drawing which shows an example of the conventional human interface apparatus.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, embodiments of the invention will be described with reference to the accompanying drawings.

FIG. 1 is a block diagram showing a schematic configuration of a human interface device according to an embodiment of the present invention. The human interface device includes a light emitting unit 1, an imaging unit 2, a control unit 3, a recording unit 4, and an image processing unit 5. The control means 3 controls on / off of the emission of light by the light emitting means 1, and controls the imaging means 2 with the light emitting means 1 turned on to control the imaging object (for example, a human finger) 6 at a predetermined time interval. Image. The recording unit 4 records a series of captured images (digital data) in time series. The image processing unit 5 reads the image captured from the recording unit 4 and calculates the three-dimensional position information (for example, the three-dimensional position information of the finger) of the imaging target 6. The calculated three-dimensional position information (hereinafter also referred to as 3D data) is output from the control means 3 to the external device 7. In the present specification, unless otherwise specified, the “finger” means a portion beyond the first joint of the finger.

  The human interface device repeats the imaging of the object and the calculation of the 3D data in this manner, thereby outputting 3D data to the external device 7 in accordance with the displacement of the object, and the external device 7 uses the 3D data for any application. (For example, computer application software). In FIG. 1, only main components are shown, and other necessary elements such as a power source are omitted.

  FIG. 2 is a schematic diagram showing an example of the light emitting means 1 and the imaging means 2 of the human interface device. In FIG. 2, the LED 8 is used as the light emitting means 1. The LED 8 and the imaging means 2 are disposed inside a spherical transparent outer shell 9 disposed on the base 11. The imaging means 2 includes a fish-eye lens 10, an image sensor (not shown), and a driving device (not shown). The plurality of LEDs 8 are arranged so as to surround the fisheye lens 10. The light emitted from the LED 8 passes through the transparent outer shell 9 and illuminates a plurality of fingers 6 located outside thereof. The imaging means 2 images the illuminated finger 6 and transfers the image data to the control means 3. Here, the fisheye lens means a lens having an angle of view of about 180 degrees or more.

  Here, the light emitted from the LED 8 is infrared light, and it is desirable to provide a filter that transmits only infrared light in the space between the transparent outer shell 9 and the photoelectric conversion element of the image sensor. The intensity distribution of light emitted from the plurality of LEDs 8 is desirably axisymmetric with respect to the optical axis of the fisheye lens 10 (z-axis in FIG. 3). That is, it is desirable that the number and arrangement of the plurality of LEDs 8 be determined so as to obtain such a light intensity distribution.

  Note that the transparent outer shell 9 is not an essential component, and may be configured such that the finger 6 directly touches the fisheye lens 10. Further, instead of the fisheye lens 10, a wide angle lens (angle of view of about 60 degrees to about 180 degrees) may be used.

  Next, the operation principle will be described. The light emitting means 1 illuminates the plurality of fingers 6 and the surrounding area. In the case of a point light source that uniformly emits light, the light intensity (irradiance) per unit area of the region facing the light source is inversely proportional to the square of the distance from the light source. Therefore, the intensity of light diffusely reflected by the finger is also inversely proportional to the distance between the light source and the finger. As a result, the pixel value of the finger area of the captured image decreases with an increase in the distance from the light source. The actual light emitting means 1 is not a point light source but emits light with a specific angular distribution, but the tendency that the irradiance decreases monotonously with the distance from the light source is the same. Therefore, the pixel value of the specific region of the imaged finger changes monotonously according to the distance from the light source. Therefore, if the relationship between the distance and the pixel value is measured in advance and stored as conversion data, the pixel value of the portion corresponding to the finger in the captured image is compared with the conversion data, The distance to the light source can be calculated. This distance corresponds to the moving radius r if the position of the light source is the origin in the polar coordinates of FIG. The polar angle θ and the azimuth angle φ can be directly specified from the captured image. That is, the azimuth angle φ is an angle formed by a straight line connecting the portion corresponding to the finger and the origin and the x axis, and the polar angle θ can be obtained from the characteristics of the fisheye lens and the distance from the center of the image. As a result, the three-dimensional position information of each finger can be obtained.

The calculation of the distance in the radial direction will be described in detail with reference to numerical examples adopted in the experiment. As the image pickup means 2, a general fisheye lens is mounted on a general CMOS camera. For the light emitting means 1, 16 bullet-type LEDs were soldered to a donut-shaped printed board. That is, these LEDs are arranged so as to surround a fisheye lens located at the center of the printed circuit board (see FIG. 2). Here, in order to eliminate the influence of external light, an LED that emits near-infrared light having a peak wavelength of 950 nm was used. Image sensors such as CCD and CMOS sensors manufactured from general silicon wafers can detect light in this wavelength band, so inserting a filter that transmits only infrared light between the image sensor and the lens makes it possible to detect visible light. The effects of external light can be completely removed. For simplicity, the following shows the result of eliminating the influence of external light by experimenting in a dark room without using this type of filter.

FIG. 4 is a graph showing the light distribution characteristics when a predetermined current is passed through the light emitting means experimentally manufactured as described above. As a measuring means, a photodiode having a sensitive area of 1 cm ² is used, and the photodiode is arranged along a semicircle having a radius of 5 cm and a radius of 10 cm in the plane including the z-axis with the center of the fisheye lens as the origin of the polar coordinates in FIG. It was moved and measured. In FIG. 4, the vertical axis represents the light intensity, and the horizontal axis represents the polar angle θ (in this case, one is positive and the opposite direction is a negative value across the z axis). From FIG. 4, it can be confirmed that the radiation intensity decreases as the distance from the light source increases. The reason why the radiant intensity does not accurately become ¼ even when the distance is doubled is that a bullet-type LED having directivity is used instead of a point light source.

  FIG. 5 is an example of an image of a hand acquired with the prototype device. A fish-eye lens C-mount is visible in the vicinity of the outer edge of the image. The inside of this ring-shaped region is the outside world imaged by the CMOS camera through the fisheye lens, and the hand is clearly shown in the center of the image. FIG. 5 is an image when the distance between the hand and the center of the fisheye lens is about 5 cm. As this distance increases, the size of the hand in the captured image decreases and the brightness also decreases.

  In order to quantify the relationship between distance and brightness, the index finger was fixed to a fixture such that the belly of the finger was facing the light source, and a plurality of images were taken while sliding on the optical bench. At this time, the center of the region corresponding to the finger was adjusted to match the center of the image on the captured image. That is, the radius r is changed on the z-axis so that the polar angle θ is 0 in the coordinate system of FIG. FIG. 6 is a graph in which the finger width and the average value of the pixel values at the center of the finger are obtained from each captured image and plotted against the radius r. The white circle represents the width of the finger, and the black circle represents the average value of the pixel values at the center of the finger. From FIG. 6, it can be seen that the width of the finger monotonously decreases as the radial distance increases. This is a simple geometric effect. It can also be seen that the average value of the pixel values, which are the brightness of the finger, also decreases monotonously.

  Accordingly, any of the curves in FIG. 6 can be used as conversion data for calculating the radial position of the finger (distance from the origin of the imaging system) from the imaged finger image. However, if the finger belly does not face the light source due to the rotation of the hand, the finger width and finger brightness on the image will change, but even in that case, the finger distance can be calculated with a certain degree of accuracy. it can. In addition, accuracy can be improved by using a combination of conversion data related to the finger width and conversion data related to the average value of the pixel values at the center of the finger.

  In the following, when differentiating the conversion data, the conversion data for associating the finger width with the distance is referred to as “first conversion data”, and the average pixel value at the center of the finger is associated with the distance. The conversion data is referred to as “second conversion data”.

  4 to 6 show experimental results when using a bullet-type LED, but an example using a chip-type LED is shown below. Eight chip LEDs having a peak wavelength of 950 nm were used, arranged at equal intervals around the fisheye lens on the same circumference, and allowed to emit light by passing a predetermined current. FIG. 7 shows light distribution characteristics obtained in the same manner as the experiment of FIG. However, the measurement photodiode was moved along a semicircle having a radius of 5 to 10 cm in a plane including the z-axis.

  From FIG. 7, it can be confirmed that the intensity of light output in an oblique direction is larger than that of FIG. Also in FIG. 7, as in FIG. 4, the light intensity monotonously decreases as the radial distance r increases, indicating that the distance can be calculated based on the size of the pixel value. .

  Further, for the light source using the chip type LED, the same experiment as that for obtaining the conversion data in FIG. 6 was repeated. The obtained result is shown in FIG. A conversion curve similar to that in FIG. 6 was obtained, but when observed closely, in FIG. 8, the change in the average value of the pixel values is steep when the distance is particularly small (near the left end of the graph) compared to FIG. I understand that. This indicates that the resolution of the distance calculated from the pixel value is higher in the graph of FIG. 8 than in the graph of FIG. However, when the distance is large (near the right end of the graph), the pixel value changes more slowly in the graph of FIG. 8 than in the graph of FIG. 6, and the resolution when calculating the distance is low.

  As described above, since the shape of the conversion curve changes depending on the light distribution characteristic of the light source, the light distribution characteristic of the light source affects the accuracy when calculating the radial distance. That is, in order to accurately calculate the distance r in the radial direction of the finger, the light distribution characteristic of the light source is important, and it is desirable that the light distribution characteristic of the light source depends only on the distance r and not on the polar angle θ. . Then, it is desirable to determine the shape of the light emitting means (for example, the arrangement of the LEDs) so that the light distribution characteristic of the light emitting means depends only on the distance r in accordance with the imaging means to be used. Conversely, it can be said that the detailed configuration of the light emitting means is a matter to be determined according to the accuracy of the distance r required according to the purpose of use, the allowable manufacturing cost, and the like.

  6 and 8 show the conversion data when the polar angle θ is equal to 0, but similar conversion data can be obtained even when the polar angle θ is not zero. Although this also depends on the specific design, in the case of the light source using the above-described chip-type LED, these conversion data substantially coincide with each other when θ <70 °. Therefore, the amount of data necessary for calculating the distance r is reduced.

  FIG. 9 is a flowchart showing an operation of acquiring finger position information using the human interface device. The operation of the present human interface device will be described with reference to FIGS.

  In step S <b> 1, the control unit 3 turns on the light emitting unit 1, performs imaging using the imaging unit 2, and records image data in the recording unit 4. As a result, an image as shown in FIG. 5 is obtained. The image to be picked up is, for example, a multi-tone gray scale image.

  In step S2, the image processing means 5 reads the image data from the recording means 4 and detects a finger area in the image. Since a known image process such as edge detection may be used for finger detection, detailed description thereof is omitted.

  In step S3, the image processing means 5 calculates the width w of each finger or a predetermined area of each finger (hereinafter also referred to as area A (refer to the oval area in FIG. 3)) from the finger area detected in step S2. An average value a of pixel values is calculated. For example, the width w and the average value a are calculated for each of five fingers.

  In step S4, the image processing means 5 uses the width w or the average value a obtained in step S3 and the conversion data read from the recording means 4 to position coordinates (r, θ, φ) of each finger. Is calculated and output as data for each finger. The conversion data is data measured in advance using the light emitting means 1 and the imaging means 2 to be used, for example, data as shown in FIGS.

  At this time, when the width w is calculated in step S3, at least a graph of the width w and the distance r (for example, the white circle data in FIGS. 6 and 8) is recorded in the recording means 4 as the first conversion data. The distance r in step S4 is calculated using this. When the average value a is calculated in step S3, at least a graph of the average value a and the distance r (for example, the black circle data in FIGS. 6 and 8) is recorded in the recording means 4 as the second conversion data. Then, using this, the distance r in step S4 is calculated. Note that the conversion data may be recorded as a function or as discrete data. If it is a function, the distance r can be calculated directly from the measured value (w or a). In the case of discrete data, the distance r corresponding to the measured value (w or a) may be calculated by an interpolation method or the like.

  The polar angle θ can be calculated from the distance (number of pixels) from the center of the image to the region A (representative point (for example, the center) of the region A) in consideration of the optical characteristics of the imaging means. For example, using a fisheye lens, the optical system may be designed so that the distance from the center of the image is proportional to the polar angle θ. Further, the azimuth angle φ is calculated as the angle of the region A (representative point of the region A) with respect to the reference axis (x axis) in the image.

  The (r, θ, φ) obtained for each finger is output to the external device 7 via a predetermined interface. The data output format is arbitrary. In the case of a serial interface, for example, the time series order of data may be associated with each finger, and (r, θ, φ) from the thumb to the little finger may be output in order.

  In step S5, it is determined whether or not the process is finished. If the process is not finished, the process returns to step S1, and the above steps S1 to S4 are repeated at a predetermined time interval. The time resolution of the change in the position of each finger depends on the performance of the imaging means 2 and the image processing means 5. For example, if an image is captured and processed at a rate of N frames / second, the change in the position of each finger can be obtained with N resolution per second.

  Since an image sensor based on CMOS technology has a high degree of freedom in the output format of an image, if a CMOS image sensor is used for the imaging means 2, for example, it is possible to output at a high speed by limiting the target finger area. is there. In addition, unlike normal person and landscape imaging, the purpose is to determine the displacement of the center of the finger, so the demand for image definition is low. This fact is also advantageous for high-speed data output, and for example, a speed exceeding 200 fps (frames / sec) can be realized. Therefore, the coordinate input speed is not inferior to that of a general mouse.

In the above description, the distance r is calculated using the width w or the average value a. However, both the width w and the average value a may be used. For example, the width w and the average value a are calculated in step S3, the distance r is obtained using the first and second conversion data in step S4, and the distance r _w obtained from the first conversion data is obtained. From the distance r _a obtained from the second conversion data, the distance r may be finally obtained as an average value (r _w + r _a ) / 2, for example. Further, the conversion data may be properly used according to the distance. For example, when the data of FIG. 6 is used, when the distance r is about 5 cm, the change in the first conversion data is steeper and the resolution is higher. The conversion data may be used, and the second conversion data may be used in other ranges.

The conversion data obtained by measuring at least once for a specific human interface device and a specific person can be used for different persons. However, since there are individual differences (gender, age, skin color, etc.) in finger shape and light reflectance, conversion data may be different even when measured with the same human interface device. Therefore, in order to obtain the distance with higher accuracy, it is desirable to perform calibration first for each person. For example, the finger is first positioned at the position of the predetermined distance r = r ₀ , and the distance is calculated by performing the processing of steps S1 to S4 once, and according to the difference between the calculated r ₁ and r ₀ , The recorded conversion data can be corrected, and the corrected conversion data can be used in subsequent processing. Method of correction, for example, to shift the conversion data according to the difference between r ₁ and r _0, there are methods such as scaling the converted data in accordance with the ratio of r ₁ and r _0.

  Further, since the width w of the finger varies depending on the type of finger (thumb, forefinger, middle finger, ring finger, little finger), the first conversion data may be recorded for each finger. For example, different conversion data may be used for conversion data for the thumb and conversion data for the little finger. At this time, the type of finger may be determined from the size of each finger detected by image processing and the mutual positional relationship.

  The first conversion data is not limited to data in which the absolute distance and the finger width are associated with each other, and may be any data that can convert the finger width into the distance. For example, it may be data in which the magnification with respect to the minimum distance (distance between the outer shell surface and the origin in FIG. 2) and the width of the finger are associated with each other.

Further, when the angle dependency of the light distribution characteristic is small, only the second conversion data obtained by measurement with θ = 0 can be used, but the light distribution characteristic is as shown in FIGS. If there is angle dependency, the second conversion data may be corrected with the light distribution characteristic and used. For example, when the second conversion data is data obtained by measurement at θ = 0, the polar angle θ of the fingertip portion is obtained (θ = θ ₁ ) before calculating the distance r, and θ = The second conversion data may be corrected with the ratio of the light intensity of the light distribution characteristics corresponding to θ ₁ and θ = 0, and the distance r may be calculated using the corrected conversion data. Further, in addition to the second conversion data measured at θ = 0, a plurality of second conversion data measured at 0 <θ <90 (degrees) may be used. In this case, the polar angle θ of the finger may be obtained before calculating the distance r, and the distance r may be calculated using a plurality of second conversion data according to the obtained angle θ ₁ .

  Alternatively, a predetermined reflector may be attached to a human fingertip and the intensity of light reflected by the reflector may be used. For example, it is possible to directly attach to the fingertip of each finger or to use a glove with a reflector attached to the fingertip portion. In this case, the conversion data may be obtained by measuring with the same reflector attached.

  Further, not only when the finger width w and the average value a of the finger pixel values are used, if the feature amount of each finger extracted from the captured image depends on the distance, the feature amount The relationship with the distance can be measured in advance and used as conversion data. For example, the feature amount may be a finger area.

  Moreover, not only a finger but a thing which reflects light to some extent and whose three-dimensional position can be displaced may be used. For example, a palm cocoon may be used. If this is recognized as a palm, it can also be used as a game interface related to fortune-telling.

  Alternatively, an image may be acquired using not only light reflection but also absorption, and the displacement of the hand or finger may be detected based on the feature amount of the image. An example of this is a vein inside the hand or finger. Detecting a palm vein pattern using the intensity of infrared light absorption and performing personal authentication based on this feature is widely used in bank ATM devices and the like. Therefore, if such a known technique is used, a vein pattern can be detected from an image as shown in FIG. If the hand is moved, the vein pattern also changes, and conversely, the displacement of the hand can be detected from the change in the vein pattern.

DESCRIPTION OF SYMBOLS 1 Light emission means 2 Imaging means 3 Control means 4 Recording means 5 Image processing means 6 Finger 7 External device 8 LED
9 Outer shell 10 Fisheye lens 11 Base

Claims (4)

A light emitting means, an imaging means, and an image processing means,
In a state where the light emitting means radiates light to the object, the imaging means detects light reflected from the object and acquires an image,
The image processing means
In the image, an image area corresponding to a specific part of the object surface is detected,
Using the detected feature amount of the image area and the conversion data, obtain a distance from a reference position to the specific portion of the object surface,
The conversion data, Ri Oh the data associating the distance between the feature amount from the reference position,
The feature quantity, human interface device, wherein the brightness der Rukoto of the image area. The object is a human hand;
The human interface device according to claim 1 , wherein the specific portion is a tip portion of a finger. The light is near infrared light;
The imaging means comprises a fisheye lens;
The image processing means obtains the polar angle and azimuth angle of the specific part in polar coordinates from the position of the specific part on the image and the optical characteristics of the imaging means,
The human interface device according to claim 2 , wherein the distance, the polar angle, and the azimuth angle are determined as three-dimensional position information of the specific portion. The image processing means
Using the polar angle dependency of the light distribution characteristic of the light emitted from the light emitting means and the polar angle of the specific part, the conversion data relating to brightness is corrected,
The human interface device according to any one of claims 1 to 3 , wherein the distance is obtained using the corrected data for conversion.