JP3875653B2 - Droplet state measuring device and state measuring method - Google Patents

Droplet state measuring device and state measuring method Download PDF

Info

Publication number
JP3875653B2
JP3875653B2 JP2003161213A JP2003161213A JP3875653B2 JP 3875653 B2 JP3875653 B2 JP 3875653B2 JP 2003161213 A JP2003161213 A JP 2003161213A JP 2003161213 A JP2003161213 A JP 2003161213A JP 3875653 B2 JP3875653 B2 JP 3875653B2
Authority
JP
Japan
Prior art keywords
droplet
cameras
light
state
camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
JP2003161213A
Other languages
Japanese (ja)
Other versions
JP2004361291A (en
Inventor
慎一 富永
正昭 川橋
淑夫 座間
Original Assignee
正昭 川橋
日本ノッズル精機株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 正昭 川橋, 日本ノッズル精機株式会社 filed Critical 正昭 川橋
Priority to JP2003161213A priority Critical patent/JP3875653B2/en
Publication of JP2004361291A publication Critical patent/JP2004361291A/en
Application granted granted Critical
Publication of JP3875653B2 publication Critical patent/JP3875653B2/en
Application status is Expired - Fee Related legal-status Critical
Anticipated expiration legal-status Critical

Links

Images

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an apparatus and method for measuring the state of a droplet such as a droplet or a bubble existing in a three-dimensional space.
[0002]
[Prior art]
An apparatus for measuring the state of a droplet such as a droplet or a bubble existing in a three-dimensional space is required in many fields.
[0003]
For example, it is necessary to control the combustion state in order to reduce CO2 and Nox contained in the exhaust gas discharged by the combustion of the internal combustion engine. Appropriate evaluation of the distribution or diffusion state of the sprayed fuel greatly contributes to the development of a fuel injection nozzle for an internal combustion engine. In addition, the distribution of droplets existing in a three-dimensional space, such as evaluation of spray characteristics of nebulizers, humidifiers, etc., measurement of the distribution and diameter of vapor rising from specific chemicals, observation of the behavior of beer and wine bubbles, etc. There is a great need to measure the diameter and diameter accurately.
[0004]
Patent Document 1 (Japanese Patent Laid-Open No. 10-90157) discloses an example of a measuring apparatus using a laser diffraction method. The measurement principle of this device is that the laser transmitter and its light receiver are exposed in the spray space, and the laser beam transmitted through the spray space is received by the light receiver to detect the scattering intensity distribution and transmittance in the spray space. Is. In general, the scattering intensity distribution is applied to a spray particle size distribution model to calculate the spray particle size distribution and the representative particle size. When the particle size distribution is calculated, the attenuation cross-sectional area of the particle group is calculated, and the average value of the volume concentration in the light flux is estimated from the measured transmittance.
[0005]
Conventionally, a method called LDV (laser Doppler flow velocity method), phase method LDV, PDPA (phase Doppler particle analysis method), etc., has been proposed to determine the position in a three-dimensional space and measure multiple particles simultaneously. Has been. The basic principle of this measurement method is that the two laser beams are “crossed” in the air to form a spatial interference fringe, and the light scattered from the droplets across the interference fringe is the same from different points. The measurement volume is observed, and the diameter of the droplet is measured from the phase difference of the measurement signal.
[0006]
Note that PIV (particle image flow velocity measurement method) is known as a measurement method for obtaining a velocity field in a plane. This measurement method is based on the assumption that the arrangement pattern created by a droplet group distributed in an observation area of a certain size does not change in the direction of flow for a certain period of time. Thus, the movement distance is calculated. Also known is SPIV (stereoparticle image velocimetry) in which this is three-dimensionalized.
[0007]
On the other hand, in recent years, the measurement space is irradiated with a sheet-like parallel laser beam (radiation sheet light), and the small droplet hit by the laser beam is out of focus due to interference between the reflected light on the droplet surface and the primary refracted light. A measurement method for analyzing the interference fringes generated in the image has been developed. This measurement method, called laser interference imaging, has interference fringes in the circular defocused image corresponding to each droplet, and there is a fixed relationship between the number of interference fringes and the droplet diameter. In particular, it is possible to measure the diameter of the droplet with high accuracy by measuring the number of the interference fringes (Non-Patent Document 1: SAE Paper No. 950457, etc.).
[0008]
However, in this measurement method, the out-of-focus image itself is circular and occupies a large area. Therefore, if the distribution density of the droplets in the space is high, the out-of-focus images overlap each other, and the individual droplets are separated and separated. There was a problem that it was difficult to measure the diameter.
[0009]
In this regard, in Patent Document 2 (Japanese Patent Application Laid-Open No. 2002-181515), by devising an optical system, interference between circular defocus images corresponding to droplets is suppressed, and analysis of defocus images is performed. The technology which made easy is disclosed. This technique compresses a circular defocused image corresponding to each droplet in one direction to form a linear image only in the other direction, so even when the distribution density of the droplets in the space is high. The respective out-of-focus images can be separated from each other, and the number of interference fringes in the out-of-focus images can be accurately counted.
[0010]
In addition, in Patent Document 2, in addition to this, the center of the linear defocused image is obtained to obtain the center position of the droplet, or the linear defocused image is subjected to Fourier transform to obtain the frequency. There is also a more specific method of obtaining the number of interference fringes in the out-of-focus image by multiplying the obtained frequency by the length of the out-of-focus image, and obtaining the diameter of the droplet based on the number of the interference fringes. It is also disclosed. Furthermore, two sets of two-dimensional frozen images are photographed at a minute interval, and the direction and distance in which a specific linear defocused image itself moves between the two sets of two-dimensional frozen images are obtained by cross-correlation calculation. A method is also disclosed.
[0011]
[Patent Document 1]
JP-A-10-90157
[Non-Patent Document 1]
SAE Paper No. 950457
[Patent Document 2]
JP 2002-181515 A
[0012]
[Problems to be solved by the invention]
However, although the measurement method according to Patent Document 1 can measure the distribution of the droplet diameter, there is a problem that information on the position of the droplet cannot be obtained. For this reason, it is impossible to obtain information on how the sprayed droplets diffuse into the space at what trajectory or speed. Further, since the laser transmitter and the light receiver are exposed in the spray space, there is a serious problem that the original flow of the spray is disturbed by the presence of the laser transmitter and the light receiver.
[0013]
Further, the measurement method called LDV, phase method LDV, PDPA, etc. measures the diameter of a droplet in a very narrow region where laser beams intersect (close to a point). It was difficult to understand the spatial relationship of droplet behavior and to analyze unsteady spray fields such as turbulent flow. Moreover, the measurement accuracy is not always sufficient.
[0014]
On the other hand, the measurement method belonging to the category of PIV can know the three-dimensional velocity distribution, but cannot measure the particle size of each droplet. In addition, since it is not originally focused on the behavior of the particle size of individual droplets, it may be difficult to detect the amount of movement of particle groups when measuring flow fields with greatly different particle velocities. . An improved method has been proposed for obtaining the particle size from the luminance profile of the droplet image, but only a rough particle size measurement result has been obtained. In addition, since the luminance profile becomes stronger in proportion to the square of the particle size, there is a possibility that the velocity distribution and the particle size are obtained as the velocity distribution of the large particles in the measurement space or averaged information. high. The problem remains that the dynamic range of the measurement is greatly reduced when the density of the droplets is high.
[0015]
The technique according to Patent Document 2 (Japanese Patent Laid-Open No. 2002-181515), which improves the laser interferometry, can determine the particle size of each droplet with high accuracy by analyzing interference fringes generated in an out-of-focus image. Although this technology has been attracting attention in recent years, there is a big problem that the position information (distribution information) and velocity information of droplets can only be captured in a two-dimensional observation field.
[0016]
In general, when examining the particle size or distribution of droplets existing in a three-dimensional space, observation in two dimensions is not sufficient. In particular, the spatial concentration of dispersed droplets floating in turbulent flow is not always uniform, and uneven density may occur depending on the spatial structure of the flow field, resulting in locally high or low concentration regions. Are known. Therefore, this laser interferometry measurement may only show local high-density or low-density droplet distribution or velocity information that occasionally happened on a specific plane in space. It was pointed out that the original distribution state or velocity information may not necessarily be reflected. However, as long as a method of obtaining interference fringes from an out-of-focus image is used, it is impossible to reconstruct a three-dimensional distribution or velocity from the obtained image information.
[0017]
Moreover, even if a technique is employed in which an out-of-focus image is optically processed to be linearized and images of overlapping droplets are separated, each droplet image is an image having a considerable length. The images are enlarged, and the images will still overlap each other depending on the droplet density. That is, as the droplet density increases, the distribution resolution must be relatively low.
[0018]
As described above, a measurement method that can accurately and simultaneously measure the particle size, three-dimensional distribution, and three-dimensional velocity of droplets such as droplets and bubbles existing in a three-dimensional space has not been developed yet. Therefore, in order to accurately measure all these factors, the actual situation is that separate measurements using different kinds of measurement methods must be performed in parallel.
[0019]
The present invention has been made in order to drastically improve the situation related to the conventional state measurement of droplets, and based on a new measurement principle, all these elements can be measured simultaneously and accurately. It is an object of the present invention to provide a droplet state measurement apparatus and a state measurement method.
[0020]
[Means for Solving the Problems]
The present invention relates to a state measuring device for a droplet for measuring the state of a droplet such as a droplet or a bubble existing in a three-dimensional space. A laser irradiation mechanism capable of irradiating the radiating sheet light, and a point obtained by irradiating each radiating sheet light to each small droplet existing in the measurement area in the radiating sheet light. A first camera that captures a group of light in a focused image from outside the range irradiated with the radiation sheet light, and a group of point light out of the range irradiated with the radiation sheet light And a second camera that captures a focused image from a different angle from the first camera, and each of the point-like groups of light captured by the first and second cameras. Based on the focus image, a drop or small droplet in the measured area The above problem can be solved by configuring the group so that it can be identified in a three-dimensional space in the form of bright spot pairs or bright spot pair groups composed of point-like light obtained for each droplet. It is a thing.
[0021]
FIG. 2 shows the relationship of the ray trajectory with respect to the focal plane P1 when the parallel laser beam LSo is irradiated to the droplet P such as a droplet or a bubble. When the parallel spherical laser beam LSo is irradiated onto the transparent spherical droplet P, the double image of the bright spot on the focal plane (two points obtained by the zero-order reflection and the first-order refraction (or second-order refraction) of the droplet P is obtained. Shaped light: bright spot pair GP). In the figure, θ is a glancing angle (scattering angle: described later). This bright spot pair GP has information on the particle size of the droplet P. However, when a large number of liquid droplets actually existing in the measurement space are imaged, in the state of the raw image, the large number of bright spot pairs GP are scattered over the entire imaging surface, and each small droplet is one by one. P cannot be identified (see FIG. 4, FIG. 7A, etc.).
[0022]
In the present invention, this bright spot pair is captured as an in-focus image (instead of a technique in which an interference fringe is obtained by deliberately imaging the bright spot pair with an out-of-focus surface as in the technique of Patent Document 2). At the same time, each droplet was captured from different angles by the first and second cameras, and a method of “searching / identifying” each single droplet by comparing the stereo images (two images) was adopted. . That is, in the present invention, a stereo image is used as a means for searching and identifying each droplet itself.
[0023]
In the present invention, innumerable scattered spot-like light is identified under the concept of two paired spot-like light spots, that is, “bright spot pairs”. Therefore, each droplet can be individually identified. Moreover, since each identified droplet is depicted in the form of a focused bright spot pair, it inherently contains particle size information (regardless of whether the particle size is large or small). I) The particle size is optically reflected correctly.
[0024]
Further, since the focused bright spot pair has almost no area, even if the droplets are densely present in the measurement region, there is almost no description interference between the droplets. Therefore, depending on the resolution of the camera or lens, it is possible to reliably identify each single droplet even in a space where small-sized droplets that could not be measured conventionally exist in high density.
[0025]
In addition, the present invention is superior in that the two focused images captured from different angles by the first and second cameras not only contribute to the identification work of each single droplet. This means that at the time of the identification, information on the three-dimensional distribution (position in three dimensions) of each droplet in the measured region is included at the same time. That is, the analysis relating to the bright spot pair of each droplet existing on the two image planes according to the present invention makes it possible to simultaneously grasp the three-dimensional distribution state of each droplet in the measurement region.
[0026]
Further, in the present invention, each droplet can be specified in the form of a pair of bright spots, so that the state at a specific time and the state after a lapse of a minute time are respectively captured by the first and second cameras. Thus, movement information of each droplet on each image plane captured by the first and second cameras can be obtained. Therefore, it is possible to calculate the three-dimensional movement trajectory and velocity information of each droplet by the mutual relationship between the two.
[0027]
That is, according to the present invention, it is possible to simultaneously acquire the particle size, the three-dimensional distribution (three-dimensional position information), the three-dimensional movement trajectory and the three-dimensional velocity (three-dimensional vector information) of each droplet, which was not possible conventionally. is there.
[0028]
By the way, the present invention has a “possibility” that can identify all the droplets existing in the measurement region in principle as described above, and as a result, all the droplets have a particle size, a distribution in three dimensions ( (Position information in three dimensions), and further, velocity distribution in three dimensions can be measured. However, the present invention does not necessarily require that all the droplets existing in the region to be measured be identified in units of individual droplets in actual measurement, but only by identifying the number or mode according to the purpose. It may be enough.
[0029]
For example, when it is desired to determine the particle size, it is not necessary to identify all the droplets in the region to be measured. In this case, limit the number of computations and the number of treatments for identification, or identify only the number that seems to be necessary by adding conditions such as identifying only those that have been captured with a clear pair of bright spots. It is only necessary to measure the particle size of only the droplets identified under the conditions.
[0030]
Also, in the measurement of the three-dimensional distribution or the measurement of the three-dimensional velocity distribution, when it is difficult to identify all the droplets due to the resolution of the first and second cameras or the processing capacity of the computer. For example, a group of bright spots formed by a plurality of droplet groups may be searched and identified as one lump. Even in this case, a plurality of droplet groups can be identified as group-level droplet groups that are assumed to have the same three-dimensional position and the same behavior (that is, the original position is different but the two-dimensional information is different). Therefore, the position information and behavior tracking information are highly reliable. Therefore, even when an image is obtained after a predetermined time has passed in order to obtain speed information, tracking is easy and sufficient information can be obtained depending on the application.
[0031]
Whether to identify a single droplet or multiple units should be selected mainly in relation to the resolution of the camera, the computing power of the computer, or the processing time and cost. In this respect as well, it can be flexibly handled according to the purpose.
[0032]
In the present invention, the droplet size, three-dimensional distribution, or three-dimensional movement trajectory and three-dimensional velocity can be measured simultaneously. However, in actual implementation, Needless to say, it is not always necessary to perform all of these measurements at the same time. Depending on the purpose, only one of the measurement items may be calculated and unnecessary calculations may be omitted.
[0033]
Various variations are conceivable for the present invention.
[0034]
For example, as the two point-like lights obtained by irradiating the droplet with the radiation sheet light, the 0th-order reflected light and the first-order refracted light of the droplet are selected so that each droplet is bright and clear. A bright spot pair. In other words, according to the present invention, for example, the second-order refracted light of a droplet can be selected as an object constituting a bright spot pair.
[0035]
Further, when measuring the diameter of the droplet, the interval between the bright spot pairs may be calculated by an autocorrelation method. A method for calculating the distance between two points by the autocorrelation method is known. This calculation has a lighter calculation burden than, for example, a method of analyzing interference fringes using Fourier transform.
[0036]
In addition, when the depth (thickness) in the width direction of the radiation sheet light is variable, it is possible to obtain a measurement region having a size that matches the measurement purpose. That is, if the depth of the radiation sheet light in the width direction is deep (thick), the measurable depth of the three-dimensional space can be increased. Can reliably track large droplets. On the other hand, when the density of small droplets is high, the number of small droplets to be imaged (the number of bright spot pairs) increases, which not only increases the burden of calculation processing for identifying each bright spot pair, but also erroneously. This increases the probability that the identification will be performed. In such a case, if the depth of the radiation sheet light in the width direction is reduced, the number of bright spot pairs to be imaged can be reduced. If the depth of the radiation sheet light in the width direction is variable, such adjustment according to the purpose can be easily performed.
[0037]
Further, the optical axes of the first and second cameras may be arranged so as to intersect with the sheet plane of the radiation sheet light at an intersecting angle other than a right angle (glazing angle and stereo angle: detailed later).
[0038]
In general, 0th-order reflected light, first-order refracted light, or second-order refracted light has different luminances. If the first and second cameras are arranged with an “angle” so that the optical axis of the camera intersects the sheet plane of the radiating sheet light at an angle other than a right angle, the difference can be reduced. As a result, the droplet can be identified more reliably.
[0039]
In this case, if the angle of view is variable, the camera arrangement can be easily adjusted so that a bright spot pair can be obtained most clearly in accordance with the characteristics of the object to be measured, such as transparency and refractive index. And the degree of freedom of setting can be increased.
[0040]
Further, when the angle (stereo angle) formed by the optical axes of the first and second cameras is variable, the resolution and depth (width) in the direction of the radiation sheet in the three-dimensional space of the measurement area The direction resolution can be easily adjusted.
[0041]
In addition, when the distance between the first and second cameras with respect to the measurement area is variable, the size of the imageable space can be easily changed in consideration of the movement of the measurement object. Become.
[0042]
In addition, when the measurement area imaged by the first and second cameras can be set at an arbitrary position in the space where the droplet exists, the presence of the droplet is traced by tracing the measurement area. Measurement of the entire three-dimensional space can be performed.
[0043]
By the way, in the present invention, the basic configuration is to identify each droplet in the measurement area based on the focused images respectively captured by the first and second cameras. Installation of cameras other than these two cameras is not prohibited, and other cameras may be provided as appropriate according to the use or purpose.
[0044]
For example, in addition to the first and second cameras, the point-like group of light is outside the range where the radiation sheet light is irradiated and is different from both the first and second cameras. A check camera that captures an in-focus image from an angle is provided. In addition to the first and second cameras, the in-focus image captured by the check camera is also referred to, and a droplet (or a droplet group) in the measurement region ) Can be identified, each droplet can be identified more accurately, and the position of the droplet in the three-dimensional space can be grasped more accurately. Become.
[0045]
The check camera treats the in-focus image captured by the check camera as equal to the in-focus image captured by the first and second cameras, and averages the numerical values obtained by the three cameras. It may be used in such a manner that it is referred to only when the correlation between the first and second cameras is questioned.
[0046]
This check camera has its own optical axis in the camera plane including the optical axes of the first and second cameras, and is installed at a position corresponding to the center of the first and second cameras. Even better. In general, when focusing only on the function of clearly capturing a bright spot pair, it is most preferable to install the camera at a position corresponding to a stereo angle of 0 degrees with respect to the radiation sheet light. Therefore, if a focused image of a small droplet at a stereo angle of 0 degrees is separately present, not only can the probability of erroneous identification be reduced, but also the amount of bright spot pairs that are clearly captured, for example, for determining the interval In the calculation by the autocorrelation method, the peak can be easily found, so that the particle size of the droplet may be calculated more accurately.
[0047]
Furthermore, in addition to the first and second cameras, a pair having two or more camera sets having the same configuration as the first and second camera sets may be provided. For example, when a set of first and second cameras having the same glancing angle and a small stereo angle is combined with a set of third and fourth cameras having a large stereo angle, the set of the first and second cameras In particular, a bright spot pair image having a high resolution in a direction parallel to the sheet surface of the radiation sheet light is obtained, and a bright spot pair image having a high resolution in the width direction of the radiation sheet light is obtained by the third and fourth camera sets. can get. As a result, compared to a case where only a pair of camera sets are provided, it is possible to identify droplets and determine the position distribution with very high accuracy in all three-dimensional directions.
[0048]
For example, if the measurement area of the first and second camera sets and the measurement area of the third and fourth camera sets are set to be continuous, a wide range of small droplets can be measured in one measurement. Distribution or movement can be continuously measured or tracked with high accuracy in focus.
[0049]
Such coordinated measurement by three or more cameras can be assumed as an advanced form of the present invention because the present invention performs identification analysis based on the focused image of a bright spot pair. It can be said that it shows potential.
[0050]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[0051]
FIG. 1 is a conceptual configuration diagram for explaining the basic principle of measurement according to the present invention.
[0052]
The measurement target in this embodiment is a droplet (small droplet) sprayed downward from the swirl nozzle N. For convenience, the horizontal direction in which the radiation sheet light LS is emitted is X direction, the vertical direction (direction in which the central axis of the swirl nozzle N extends) is Y direction, and the horizontal direction (radiation) is perpendicular to both the X direction and the Y direction. The width direction of the sheet light LS) is defined as the Z direction. The origin in each direction is the nozzle of the swirl nozzle N.
[0053]
Referring to FIG. 1, a state measurement apparatus 10 includes a laser irradiation mechanism 12, a first camera 14, a second camera 16, and a computer (calculation means: not shown) as main components.
[0054]
The laser irradiation mechanism 12 irradiates a sheet-shaped radiation sheet light LS having a thin width (thickness W) from a light source 12A to a three-dimensional space in which droplets exist, and is known per se. .
[0055]
The radiation sheet light LS is fan-shaped light that spreads symmetrically in the vertical plane (in the XY plane) around the horizontal line x0 in the X direction with the light source 12A of the laser irradiation mechanism 12 as a base point. The thickness W in the Z direction of the radiation sheet light LS becomes the substantial depth in the Z direction of the measurement region S in this embodiment as it is. That is, when the thickness W is set large, the depth of the measurement region S in the Z direction can be increased. Conversely, when the thickness W is set to be small, the number of droplets irradiated with the radiation sheet light LS is reduced, so that an image with little interference (easily identified) between the droplets can be obtained. Therefore, qualitatively, when the movement of the liquid droplet is large, a deep measurement area S is secured by increasing the thickness W, and when the density of liquid droplets is high, the thickness W of the radiation sheet light LS is secured. A more desirable measurement result can be obtained by reducing the number of droplets to be imaged by decreasing the value of.
[0056]
Note that the X-direction and Y-direction boundaries of the measurement target region S are determined as overlapping portions of the respective imaging regions of the first and second cameras.
[0057]
When the radiating sheet light is irradiated to each droplet existing in the measurement region S, as described above, two point-shaped lights (0th-order reflected light and first-order refracted light) for each droplet ( A bright spot pair) is obtained (see FIG. 2).
[0058]
The first camera 14 receives the radiation sheet light LS so that the group of bright spots obtained for each droplet by irradiating the droplet with the radiation sheet light LS can be captured by the focused image. Arranged outside the irradiated range. On the other hand, the second camera 16 is arranged so that the same group of bright spot pairs is outside the range irradiated with the radiation sheet light LS and is captured by a focused image from a different angle from the first camera. The two cameras 14 and 16 are CCD cameras and are arranged in a so-called stereo PIV arrangement.
[0059]
More specifically, the optical axes 14A and 16A of the first and second cameras 14 and 16 intersect with the sheet plane of the radiation sheet light LS at an angle θ other than a right angle in the XZ plane, respectively. Yes. This intersection angle θ is referred to herein as a “glare angle”. The reason why the glancing angle θ is set to an angle other than 90 degrees is to reduce the difference in luminance between the 0th-order reflected light and the first-order refracted light as much as possible so that both have substantially the same luminance. The glancing angle θ is preferably made variable because the optimum value varies depending on characteristics such as the transparency and refractive index of the droplet.
[0060]
The angle α formed by the optical axes 14A and 16A of the first and second cameras 14 and 16 is also variable. This angle α is referred to herein as a “stereo angle”. The setting of the stereo angle α (more specifically, the setting of the inclination angle α / 2 from the horizontal plane corresponding to ½ thereof) relates to the identification of the droplet in the three-dimensional space of the measurement region S, and the radiation sheet light It affects the adjustment of the resolution in the sheet surface direction (XY direction) and the resolution in the depth direction (Z direction) of LS. If the stereo angle α is set small, the resolution in the sheet surface direction can be increased, and if the stereo angle α is set large, the resolution in the depth direction (Z direction) can be increased.
[0061]
The distances d1 and d2 (d1 = d2) with respect to the measurement area S of the first and second cameras 14 and 16 are also variable. This is because the size of the imageable space can be easily changed in consideration of the movement of the measurement target.
[0062]
Note that the camera plane including the optical axes 14A and 16A of the first and second cameras 14 and 16 is the coordinate axis corresponding to the direction in which the radiation sheet light LS spreads (in this embodiment, the coordinate axis in the Y direction). Parallel. The first and second cameras 14 and 16 are symmetrically opposed to the radiation sheet light LS, and each has the same glancing angle θ and the inclination angle from the same horizontal plane (corresponding to 1/2 of the stereo angle α). The distances d1 and d2 with respect to the measurement region S are also the same.
[0063]
With the above configuration, the first and second cameras 14 and 16 can set their measurement area S at an arbitrary position in the radiation sheet light LS. Therefore, for example, as will be described later, the irradiation direction of the radiation sheet light LS to the space where the droplet exists is variable, and the measurement is repeated so as to trace the measurement target region S, resulting in the presence of the droplet. The state of the droplet can be measured in the entire three-dimensional space.
[0064]
FIG. 3 shows an example of the instantaneous image of the spray sprayed from the nozzle and the measured region S (A, B). The centers of the measured areas A and B are 20 mm in the X direction, 50 mm in the Y direction, 35 mm in the X direction, and 70 mm in the Y direction, respectively, when viewed from the center of the injection port Na of the swirl nozzle N.
[0065]
FIGS. 4A and 4B show images of spraying on the focal plane of the measurement area B and samples of bright spot pairs of droplets. These images are recorded by the first and second cameras 14 and 16 that are three-dimensionally configured with respect to the measurement region B.
[0066]
5A and 5B show a certain bright spot pair and an autocorrelation function pattern of the image. The interval between the zero-order peak and the primary peak of the correlation value corresponds to the interval L between the bright spot pairs. If the distance between the bright spot pairs is known, the particle size of the droplet can be calculated based on the well-known formula (1).
[0067]
[Expression 1]
[0068]
Here, θ is the glancing angle, and m is the refractive index of the droplet.
[0069]
Actually, the particle diameter of the droplet was calculated from the equation (1) and found to be 153.5 [μm].
[0070]
FIG. 6 shows an analysis example of the result of measuring the particle diameters of the individual droplets identified in the region A to be measured. In FIG. 6, a histogram of the number of droplets with respect to the particle size of each droplet in the measurement area A is shown. From FIG. 6, it can be seen that, in the measurement area A, the diameter of the droplets increases and the number of droplets decreases. The maximum number of droplets is recorded at a droplet diameter = 60 [μm].
[0071]
If the same measurement and analysis are performed on the droplets in the measurement region B, the difference in the number of droplets with respect to the particle size of the droplets between the measurement regions A and B is clarified.
[0072]
On the other hand, individual droplets identified in the measurement areas A and B have three-dimensional position information as they are in the configuration of the present embodiment. This position information is nothing but a three-dimensional distribution of droplets sprayed from the ejection port (three-dimensional position information of individual droplets). Of course, how to further analyze the information of the obtained three-dimensional distribution is not limited to the above, and various methods of analysis are possible. The above analysis example is also an example.
[0073]
Furthermore, when two or more sets of images taken by the first and second cameras 14 and 16 are obtained at predetermined time intervals, information on the movement of each droplet in the three-dimensional space at each time point can be obtained. The movement trajectory of each droplet can be grasped. It is also possible to obtain a three-dimensional velocity distribution of each droplet from the relationship between the photographing time interval and the moving distance.
[0074]
FIG. 7A shows a raw image in the region A to be measured, and FIG. 7B shows a pattern example after analyzing the velocity distribution. In this example, a plurality of droplets are collectively identified in the measurement region A, and the velocity distribution of the bright spot pair group corresponding to the plurality of droplet groups is obtained. Here, only the velocity distribution in the XY direction is shown, and the velocity distribution in the Z direction is not drawn. In actual measurement, the velocity distribution in the Z direction is expressed by a color layer display.
[0075]
As described above, according to the present invention, each droplet can be individually identified based on the two images captured by the first and second cameras 14 and 16, but in this way, a plurality of droplet groups can be identified. Even if these are identified together, a corresponding effect can be obtained. The size of the droplet group to be captured together may be set in consideration of cost, processing time, and the like.
[0076]
By the way, in the present invention, each droplet in the measurement region S is identified based on images taken from different angles. Therefore, various additional effects can be obtained by increasing the number of cameras to shoot.
[0077]
FIG. 8 shows an example of an embodiment in which another camera is provided in addition to the first and second cameras 14 and 16. FIG. 8A shows an example in which a center camera (check camera) 30 is further provided in addition to the first and second cameras 14 and 16. The center camera 30 corresponds to the middle of the respective angles that the first and second cameras 14 and 16 have with respect to the radiation sheet light LS on the camera plane of the first and second cameras 14 and 16. It is installed at an angle, that is, an angle corresponding to a stereo angle of 0 degrees.
[0078]
With this configuration, in addition to the first and second cameras, the in-focus image captured by the center camera 30 is also referred to so that each droplet in the measured region S can be identified in the form of a bright spot pair. Become. As a result, each droplet can be identified more accurately and more accurately, and the particle size can be measured and the position of the droplet in the three-dimensional space can be accurately identified.
[0079]
For example, in the case of particle size measurement, only when the center camera 30 captures the same bright spot pair image in the space where a specific droplet is estimated to be present by the first and second cameras 14 and 16. If the software is configured so that it can be used for particle size measurement, assuming that two spot-like light spots are indeed present, it is possible to use them for particle size measurement. It is possible to prevent the diameter measurement.
[0080]
The center camera 30 treats the focused image captured by the center camera 30 on an equal basis with the focused images captured by the first and second cameras, and averages the numerical values obtained by the three cameras. It may be used in such a manner that it may be referred to only when the correlation between the first and second cameras 14 and 16 is questioned.
[0081]
Further, as described above, since the focused image captured by the center camera 30 is an image at a stereo angle of 0 degrees, there is a high possibility that a bright spot pair is captured more clearly. The measurement may be performed based on the bright spot pair captured by the center camera 30.
[0082]
On the other hand, in FIG. 8B, in addition to the first and second cameras 14 and 16, the third and fourth cameras 26 and 28 having the same configuration as the first and second camera sets are shown. The camera set is arranged on the same camera plane. In this example, the stereo angle α1 of the first and second cameras 14 and 16 sets is set to be smaller (a value closer to zero) with the same glancing angle θ, while the third and fourth cameras 26 and 28 are set. The stereo angle α2 is set to be larger (a value closer to 180 °). As a result, the set of the first and second cameras 14 and 16 obtains a bright spot pair image having high resolution, particularly in a direction parallel to the sheet surface of the radiation sheet light LS (XY direction). By setting the fourth cameras 26 and 28, a bright spot pair image having a high resolution in the width direction (Z direction) of the radiation sheet light LS is obtained. As a result, compared to a case where only a pair of camera sets are provided, it is possible to identify droplets and determine the position distribution with very high accuracy in all three-dimensional directions.
[0083]
As described above, when the third and fourth camera sets are also provided, the measurement areas of the first and second camera sets and the measurement areas of the third and fourth camera sets are arranged to be continuous. It is also possible to do. With this installation, even fast moving droplets can be measured or tracked continuously without being off the screen in a single measurement.
[0084]
There are many examples of cooperative measurement using three or more cameras. The usage example of the camera to be added is not limited to the above example.
[0085]
Finally, an example of the configuration of a specific apparatus used when actually implementing the present invention will be briefly described.
[0086]
9 is an overall schematic front view of the state measuring apparatus 10, FIG. 10 is a perspective view thereof, FIG. 11 is an overall schematic perspective view of a camera installation mechanism, and FIG. For ease of understanding, the same reference numerals are used as they are for members having the same functions as those described so far.
[0087]
This state measuring device 10 is for measuring the diffusion state of droplets sprayed from the swirl nozzle N, and includes a laser irradiation mechanism 12, a first camera 14, a second camera 16, and a computer (calculation means) 18a, 18b is provided as a main component.
[0088]
The laser irradiation mechanism 12 irradiates a sheet-shaped radiation sheet light LS having a thin width (thickness W) onto a three-dimensional space in which droplets exist. In this embodiment, a double pulse Nd: YAG laser is used. (Λ = 532 nm, maximum output 50 mJ / pulse) is used.
[0089]
In this embodiment, the thickness W of the radiation sheet light LS is set to 1 mm.
[0090]
Referring to FIGS. 11 and 12, a camera installation mechanism CS for installing the first and second cameras 14 and 16 is a turntable 31 that can rotate in the horizontal plane (XZ plane) together with the laser irradiation mechanism 12. It is arranged on the top, and is mainly composed of a base body 32, an L-shaped angle 34, and support arms 36 and 37. The turntable 31 can also move up and down.
[0091]
The base body 32 is fixed on the turntable 31 along the direction (X direction) in which the radiation sheet light LS is irradiated, and the fixing position of the turntable 40 is variable in the range of M in the X direction. A groove 42 is provided.
[0092]
The L-shaped angle 34 is rotated on the turntable 40 in the XZ plane so that the glancing angle θ can be adjusted and set. The L-shaped angle 34 also has a function of making the measured region S variable in the X direction as a result of the turntable 40 itself being slidable along the groove 42 of the base body 32. Support arms 36 and 37 for supporting the first and second cameras 14 and 16 are attached to the upright 34A of the L-shaped angle 34 so as to be rotatable in the camera plane. The support arms 36 and 37 themselves can be moved back and forth with respect to the radiation sheet light LS (measurement region S) (or the first and second cameras 14 and 16 are relatively movable with respect to the support arms. As a result, the distances d1 and d2 with respect to the measurement area S of the first and second cameras 14 and 16 are variable, and the size of the imageable space is considered in consideration of the movement of the measurement object. It can be easily changed.
[0093]
The rotation angles of the L-shaped angle 34 and the support arms 36 and 37 are detected by the encoders 44, 45 and 46, respectively, so that the current glaring angle θ and stereo angle α (tilt angle α / 2) can be confirmed. Yes. Further, if necessary, it is also possible to automatically set the glancing angle θ and the stereo angle α using a linear motor (not shown) based on the detected value.
[0094]
As apparent from FIG. 11, in this state measuring apparatus 10, the entire camera support mechanism CS of the swirl nozzle N is moved by rotating or vertically moving the turntable 31 while maintaining the positional relationship of the optical system. It can be rotated 360 degrees with respect to the central axis (Y-axis), and state measurement in all directions can be performed.
[0095]
In the initial setting for manual setting or automatic setting, as shown in FIG. 13, the laser pointer 53 is applied to the positioning bar 52 attached to the mounting head 50 of the swirl nozzle N and designated. The area to be measured S is adjusted while viewing the image so that the marker is at the position.
[0096]
Further, as shown in FIG. 14, the rotating table 40 is rotated and positioned by releasing the laser pointer 55 from the laser light source 12 </ b> A, mounting a mirror 57 whose mounting angle is adjusted on the nozzle mounting head 50, and Is adjusted so as to be the highest.
[0097]
9 and 10, reference numeral 74 denotes a work desk, and 76 denotes a test fluid supply mechanism. The test fluid supply mechanism 76 includes a screw spindle pump 80 that sends out a fluid, a water pressure control panel 82 that controls the pressure, a test fluid recovery tank 84, a pipe 86, and the like. Reference numeral 90 denotes a pulse generator for sending a signal to the laser light source 12A, and 92 denotes a cylindrical lens attached to the laser light source 12A. Reference numeral 94 denotes a honeycomb rectifying board for suppressing the influence (rewinding) of the droplets in the spray field from below.
[0098]
The first and second cameras 14 and 16 are 1008 × 1016 pixel CCD cameras, the focal depth is 12.3 mm in this example, and the aperture is f = 16. As a result, a depth of field sufficient for both cameras to record a focused image is obtained within the thickness W of the radiation sheet light LS.
[0099]
In this embodiment, the glancing angle θ is set to 70 degrees. The variable range is 50 to 80 degrees. In practice, about 65 to 75 degrees is often the optimum range of the glancing angle θ.
[0100]
Further, in this example, the stereo angle α is 50 degrees (inclination of 25 degrees up and down with respect to the horizon), but a range of 20 to 60 degrees (inclination of 10 to 30 degrees with respect to the horizon). It is variable.
[0101]
With the above configuration, the first and second cameras 14 and 16 can set their measurement area S at an arbitrary position in the radiation sheet light LS. Furthermore, since the irradiation direction of the radiation sheet light LS with respect to the space where the droplet exists is also variable by the rotation and vertical movement of the turntable 31, the measurement is repeated by tracing the space where the droplet exists. As a result, the three-dimensional state in the entire range where the droplet exists can be implemented including the measurement of the particle size of the droplet.
[0102]
It should be noted that a support mechanism having the same configuration can be employed when a separate camera is added in addition to the first and second cameras 14 and 16.
[0103]
However, in the present invention, there is no particular limitation on the specific support mechanism for supporting and installing each camera including the first and second camera support mechanisms.
[0104]
【The invention's effect】
According to the present invention, it is possible to simultaneously and accurately measure the particle size of a droplet such as a droplet or a bubble existing in a three-dimensional space, a three-dimensional distribution state, and a three-dimensional velocity distribution. Excellent effect is obtained.
[Brief description of the drawings]
FIG. 1 is a conceptual configuration diagram for explaining the basic principle of measurement according to the present invention.
FIG. 2 is an optical characteristic diagram showing the relationship of the ray trajectory with respect to the focal plane in droplet image formation.
FIG. 3 is a composite diagram in which a display of a measurement area is inserted into an instantaneous image of spray sprayed from a nozzle.
FIG. 4 is an image showing a spray visualized spray on a focal plane of a measurement area B and a sample of a bright spot pair of droplets;
FIG. 5 is a pattern diagram of a certain bright spot pair and an autocorrelation function of the image.
FIG. 6 is a histogram showing an analysis example of the result of measuring the particle size of each droplet identified in the measurement area A
FIG. 7A is a raw image of the measurement area Sa, and FIG. 7B is a velocity pattern diagram after analyzing the velocity distribution.
FIG. 8 is a schematic perspective view showing an example of an embodiment in which another camera is provided in addition to the first and second cameras.
FIG. 9 is an overall schematic front view showing a specific configuration of the state measuring apparatus according to the embodiment of the present invention.
FIG. 10 is a perspective view thereof.
FIG. 11 is an overall schematic perspective view of the camera installation mechanism.
FIG. 12 is an enlarged perspective view of the main part of the same.
FIG. 13 is an explanatory diagram when a measurement area is set by a camera installation mechanism using a positioning bar.
FIG. 14 is an explanatory diagram of an example of a specific method for adjusting the glancing angle.
[Explanation of symbols]
10 ... State measuring device
12 ... Laser irradiation mechanism
14 ... 1st camera
16 ... second camera
18a, 18b ... Computer (calculation means)
26 ... Third camera
28 ... 4th camera
30 ... Center camera
LS ... Radiation sheet light
W ... Radiation sheet light thickness
S (A, B) ... area to be measured
θ ... Gaze angle
α ... stereo angle

Claims (14)

  1. In the droplet state measurement device for measuring the state of a droplet such as a droplet or a bubble existing in a three-dimensional space,
    A laser irradiation mechanism capable of irradiating a thin sheet-shaped radiation sheet light to a space where a droplet to be measured exists,
    The radiating sheet light irradiates a group of two point-like lights obtained for each droplet by irradiating the radiating sheet light to each droplet existing in the measurement area in the radiating sheet light. A first camera that captures a focused image from outside the range,
    A second camera that captures the point-like group of light with a focused image from an angle different from that of the first camera that is outside the range irradiated with the radiation sheet light;
    Based on the focused image of each of the point-like groups of light captured by the first and second cameras, two droplets or groups of droplets in the measurement area are determined for each droplet. A droplet state measuring apparatus configured to be identifiable in a three-dimensional space in the form of a bright spot pair or a bright spot pair group composed of individually obtained point-like lights.
  2. In claim 1,
    An apparatus for measuring a state of a droplet, wherein two or more sets of captured images of a point-like group of light by the first and second cameras can be acquired at a predetermined time interval.
  3. In claim 1 or 2,
    A droplet state measuring device characterized in that zero-order reflected light and first-order refracted light of the droplet are selected as two point-like lights obtained by irradiating the droplet with the radiation sheet light. .
  4. In any one of Claims 1-3,
    An apparatus for measuring a state of a droplet, wherein the diameter of the droplet is measured by analyzing an interval between the pair of bright spots by an autocorrelation method.
  5. In any one of Claims 1-4,
    2. A droplet state measuring device, wherein a depth of the radiation sheet light in a width direction is variable.
  6. In any one of Claims 1-5,
    An optical state of each of the first and second cameras intersects with a sheet plane of the radiation sheet light at an angle other than a right angle.
  7. In claim 6,
    An apparatus for measuring a state of a droplet, wherein an intersection angle between each optical axis of the first and second cameras and a sheet plane of radiation sheet light is variable.
  8. In any one of Claims 1-7,
    An apparatus for measuring a state of a droplet, wherein an angle formed by optical axes of the first and second cameras is variable.
  9. In any one of Claims 1-8,
    A droplet state measuring apparatus, wherein a distance between the first and second cameras with respect to the measurement area is variable.
  10. In any one of Claims 1-9,
    An apparatus for measuring a state of a droplet, wherein the measurement area photographed by the first and second cameras can be set at an arbitrary position in a space where a droplet exists.
  11. In any one of Claims 1-10,
    In addition to the first and second cameras, the point-like group of light is outside the range where the radiation sheet light is irradiated and is different from any of the first and second cameras. It has a check camera that captures a focused image from an angle,
    In addition to the first and second cameras, the in-focus image captured by the check camera is also referred to so that a droplet or a group of droplets in the measurement area can be identified in a three-dimensional space. A state measurement device for a droplet characterized by the above.
  12. In claim 11,
    The check camera has its own optical axis in a camera plane including the optical axes of the first and second cameras, and is installed at a position corresponding to the center of the first and second cameras. A characteristic state measurement device for small droplets.
  13. In any one of Claims 1-12,
    In addition to the first and second cameras, at least a pair of camera sets having the same configuration as the first and second camera sets are provided side by side.
  14. In the droplet state measurement method for measuring the state of a droplet such as a droplet or a bubble existing in a three-dimensional space,
    A procedure for irradiating a thin sheet-shaped radiation sheet light with a laser irradiation mechanism to a space where a droplet to be measured exists,
    By irradiating each droplet present in the measurement area in the radiation sheet light with the radiation sheet light, a group of two point-like lights obtained for each droplet is obtained with respect to the radiation sheet light. The procedure to capture simultaneously as two focused images visualized from different angles,
    Based on the two focused images, a pair of bright spots or bright spots composed of point-like light that is obtained for each droplet or group of droplets in the measurement area for each droplet. A procedure for identifying in a pairwise manner;
    A method for measuring the state of a droplet, comprising:
JP2003161213A 2003-06-05 2003-06-05 Droplet state measuring device and state measuring method Expired - Fee Related JP3875653B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2003161213A JP3875653B2 (en) 2003-06-05 2003-06-05 Droplet state measuring device and state measuring method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2003161213A JP3875653B2 (en) 2003-06-05 2003-06-05 Droplet state measuring device and state measuring method

Publications (2)

Publication Number Publication Date
JP2004361291A JP2004361291A (en) 2004-12-24
JP3875653B2 true JP3875653B2 (en) 2007-01-31

Family

ID=34053736

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2003161213A Expired - Fee Related JP3875653B2 (en) 2003-06-05 2003-06-05 Droplet state measuring device and state measuring method

Country Status (1)

Country Link
JP (1) JP3875653B2 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107449696A (en) * 2017-07-13 2017-12-08 华南农业大学 A kind of Pcnten-1 yne-4 detection means in changeable direction

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007263876A (en) * 2006-03-29 2007-10-11 Miyazaki Prefecture Calibration method in laser diffraction/scattering type particle size distribution measurement method, and measuring method of volume concentration of bubble in liquid
JP4774517B2 (en) * 2006-04-28 2011-09-14 国立大学法人埼玉大学 Particle measuring apparatus and method
US9029085B2 (en) 2007-03-07 2015-05-12 President And Fellows Of Harvard College Assays and other reactions involving droplets
JP5224756B2 (en) * 2007-09-19 2013-07-03 学校法人同志社 Droplet particle imaging analysis system and analysis method
WO2009085215A1 (en) * 2007-12-21 2009-07-09 President And Fellows Of Harvard College Systems and methods for nucleic acid sequencing
WO2010080134A1 (en) 2008-12-19 2010-07-15 President And Fellows Of Harvard College Particle-assisted nucleic acid sequencing
JP5791621B2 (en) 2009-10-27 2015-10-07 プレジデント アンド フェローズ オブ ハーバード カレッジ droplet generation technology
JP5924077B2 (en) * 2012-03-30 2016-05-25 ソニー株式会社 Fine particle sorting device and method for determining orbit direction in fine particle sorting device
EP3065712A4 (en) 2013-11-08 2017-06-21 President and Fellows of Harvard College Microparticles, methods for their preparation and use

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107449696A (en) * 2017-07-13 2017-12-08 华南农业大学 A kind of Pcnten-1 yne-4 detection means in changeable direction
CN107449696B (en) * 2017-07-13 2019-06-28 华南农业大学 A kind of Pcnten-1 yne-4 detection device in changeable direction

Also Published As

Publication number Publication date
JP2004361291A (en) 2004-12-24

Similar Documents

Publication Publication Date Title
Prasad et al. Stereoscopic particle image velocimetry applied to liquid flows
US6144453A (en) System and method for three-dimensional inspection using patterned light projection
Kawaguchi et al. Size measurements of droplets and bubbles by advanced interferometric laser imaging technique
JP3975892B2 (en) Position measurement system
CA1241721A (en) Panel surface flaw inspection
JP5189725B2 (en) Motion tracking method and system using interference pattern
US4540283A (en) Apparatus and method for determining the size and velocity of particles, droplets, bubbles or the like using laser light scattering
DE69738493T2 (en) Device and method for surface contour measurement
Maas et al. Particle tracking velocimetry in three-dimensional flows
US20030142862A1 (en) Stereoscopic three-dimensional metrology system and method
JP2005025415A (en) Position detector
US8670114B2 (en) Device and method for measuring six degrees of freedom
CN100592029C (en) Ranging apparatus
JPWO2008072369A1 (en) Measuring apparatus and measuring method
US4311383A (en) Method for determination of velocity and direction of motion of moving object by speckle patterns and apparatus therefor
CA1059752A (en) Gauging surfaces by remotely tracking multiple images
Nishino et al. Stereo imaging for simultaneous measurement of size and velocity of particles in dispersed two-phase flow
US6862097B2 (en) Three-dimensional shape measuring method, and three-dimensional shape measuring apparatus
Maeda et al. Novel interferometric measurement of size and velocity distributions of spherical particles in fluid flows
CN101140196B (en) Measurement mechanism of lens focal length, measurement method and thereof and optical quality evaluation method
Bachalo Method for measuring the size and velocity of spheres by dual-beam light-scatter interferometry
JP2012521005A (en) Optical gauge and three-dimensional surface profile measuring method
EP0144310B1 (en) A laser-doppler-apparatus for determining the size of moving spherical particles in a fluid flow
Bröder et al. An advanced LIF-PLV system for analysing the hydrodynamics in a laboratory bubble column at higher void fractions
US7205521B2 (en) Speckle based sensor for three dimensional navigation

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20050107

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20060825

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20061010

R150 Certificate of patent or registration of utility model

Free format text: JAPANESE INTERMEDIATE CODE: R150

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20061026

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20101102

Year of fee payment: 4

S531 Written request for registration of change of domicile

Free format text: JAPANESE INTERMEDIATE CODE: R313532

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20111102

Year of fee payment: 5

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20121102

Year of fee payment: 6

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20131102

Year of fee payment: 7

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

LAPS Cancellation because of no payment of annual fees