EP1225566A2 - Procédé et dispositif de focalisation en transmission et de formation de faisceau - Google Patents

Procédé et dispositif de focalisation en transmission et de formation de faisceau Download PDF

Info

Publication number
EP1225566A2
EP1225566A2 EP02000322A EP02000322A EP1225566A2 EP 1225566 A2 EP1225566 A2 EP 1225566A2 EP 02000322 A EP02000322 A EP 02000322A EP 02000322 A EP02000322 A EP 02000322A EP 1225566 A2 EP1225566 A2 EP 1225566A2
Authority
EP
European Patent Office
Prior art keywords
line data
scan
ultrasonic
transmit
delay profiles
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP02000322A
Other languages
German (de)
English (en)
Other versions
EP1225566B1 (fr
EP1225566A3 (fr
Inventor
Moo Ho Bae
Jae Bum Hong
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Medison Co Ltd
Original Assignee
Medison Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR10-2001-0088786A external-priority patent/KR100441067B1/ko
Application filed by Medison Co Ltd filed Critical Medison Co Ltd
Publication of EP1225566A2 publication Critical patent/EP1225566A2/fr
Publication of EP1225566A3 publication Critical patent/EP1225566A3/fr
Application granted granted Critical
Publication of EP1225566B1 publication Critical patent/EP1225566B1/fr
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10K—SOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR
    • G10K11/00—Methods or devices for transmitting, conducting or directing sound in general; Methods or devices for protecting against, or for damping, noise or other acoustic waves in general
    • G10K11/18—Methods or devices for transmitting, conducting or directing sound
    • G10K11/26—Sound-focusing or directing, e.g. scanning
    • G10K11/34—Sound-focusing or directing, e.g. scanning using electrical steering of transducer arrays, e.g. beam steering
    • G10K11/341—Circuits therefor
    • G10K11/346—Circuits therefor using phase variation
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/52—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S15/00
    • G01S7/52017—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S15/00 particularly adapted to short-range imaging
    • G01S7/52023—Details of receivers
    • G01S7/52036—Details of receivers using analysis of echo signal for target characterisation
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/52—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S15/00
    • G01S7/52017—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S15/00 particularly adapted to short-range imaging
    • G01S7/52046—Techniques for image enhancement involving transmitter or receiver

Definitions

  • the present invention relates to a transmit-focusing and beam-forming apparatus and method in an ultrasonic imaging system.
  • the invention relates to an apparatus for transmit-focusing and beam-forming and the method thereof, which forms adaptive scan-line data by applying a plurality of delay profiles.
  • an ultrasonic imaging system adopting a phased array includes array of a plurality of transducers.
  • Conventional transmit-focusing and beam-forming apparatuses are shown in Fig. 1A and Fig. 1B respectively.
  • these systems include a plurality of channels, and each channel includes transmitting and receiving units connected to each of the transducers.
  • the transducers of the transmitting unit fire a group of ultrasonic pulses into a target object, for instance, a human body.
  • a delay profile of the ultrasonic pulses is determined in a way that the transmitted pulses interfere each other positively at the predetermined point, and each transducer outputs pulses according to the predetermined delay profile at a proper time.
  • each of delay circuits 1 applies an appropriate transmitting delay to each of the transducer elements of the transmitting unit. Setting the applied delay as a starting point, transmitting waveforms are read from waveform table 3 and amplified by power amplifier 5. Amplified pulses are provided to each of the transducer elements and the ultrasonic pulses are transmitted.
  • the ultrasonic image is generated by beam-forming the reflected signals. Referring to Fig. 1B, a beam-forming of the reflected signals is explained.
  • the ultrasonic energy reflected off the target object returns to each array device with a time variation.
  • the received signals are converted into digital signals by A/D converter 20 and temporarily stored in memory (FIFO) 20. Then, the signals are beam-formed.
  • the digital signals stored in memory 30 are delayed by delay controller 40, focused on a predetermined point and added together. The delay is continuously calculated so that the focal point moves along the depth direction.
  • the added signals go through memory (FIFO) 50 and demodulator 60 and are displayed as an ultrasonic image on display device 70.
  • the delay profiles applied in a transmit-focusing and beam-forming are determined on the basis of the propagation velocity of the ultrasonic pulses in an object medium. That is, the delay profiles applied in the transmit-focusing and beaming forming are generated on the basis of the round-trip time that is required for the ultrasonic pulses to propagate and return.
  • the propagation velocity of the transmitted ultrasonic pulses depends on the medium characteristics where the ultrasonic pulses propagate. For instance, when the medium is a human body, the propagation velocity of the ultrasonic is 1460 m/s at the grease, 1555 m/s at the liver, 1560m/s at the blood, and 1600 m/s at the muscles.
  • the medium through which the ultrasonic pulses propagate is not uniform, and the propagation velocity can be changed along the depth of the medium.
  • the conventional transmit-focusing and beam-forming methods assume that the medium is uniform, they do not reflect the actual delay value that the ultrasonic pulses suffer as the medium changes. This results in errors of the transmit-focusing and the beam-forming.
  • the transmit-focusing and the beam-forming errors occur because the conventional ultrasonic imaging system assumes that the medium is uniform. Consequently, the image becomes distorted because of these errors.
  • the present invention provides an ultrasonic system that divides the object region, determines the accuracy of the beam-forming by using a criterion, and generates scan-line data suitable for each of the divided areas.
  • a method of an adaptive beam-forming comprises the steps of transmit-focusing ultrasonic signals upon an object region to be ultrasonic imaged by means of a plurality of transducers; (a) transmit-focusing ultrasonic signals upon an object region to be ultrasonic imaged by means of a plurality of transducers; (b) receiving the ultrasonic signals reflected off said object region at said plurality of transducers, and beam-forming the received ultrasonic signals in accordance with associated delay profiles to obtain multiple sets of preliminary scan-line data; and (c) combining said multiple sets of preliminary scan-line data into scan-line data.
  • the combining step of the adaptive beam-forming method comprises the steps of: d. dividing said object region to be ultrasonic imaged into a plurality of determining areas; e. selecting one of said multiple sets of preliminary scan-line data in each of said determining areas divided in said step d; and f. combining said selected scan-line data to form an adaptive scan-line data.
  • a method of an adaptive transmit-focusing comprises the steps of: transmit-focusing a group of ultrasonic pulses based upon a plurality of delay profiles; and transmitting said transmit-focused group of ultrasonic pulses.
  • the step of transmit-focusing said group of ultrasonic pulses of the adaptive transmit-focusing method comprises, for generating each ultrasonic pulse of said group of ultrasonic pulses, the steps of: applying each of said plurality of delay profiles, the steps of; generating a plurality of waveforms from wave form data, each of said waveforms corresponding to each of said plurality of delay profiles; and multiplexing said generated plurality of waveforms.
  • the adaptive transmitting focusing and beam forming methods of the present invention it is possible to reduce transmit-focusing and beam-forming errors generated from applying a single pattern of delay profile, and provide clear ultrasonic images.
  • the present invention is able to beam-form accurately by applying delay profiles generated on the basis of the actual medium characteristics, even though the medium characteristics change.
  • the propagation velocity along the depth of the medium may not be constant at all positions. Accordingly, it would be very difficult to generate delay profiles reflecting all changes of the propagation velocity in accordance with medium variations, and to performing a beam-forming by using the generated delay profiles.
  • the medium of an object region to be imaged may be roughly grouped into several materials. Once materials are known, it is possible to calculate the delay time that is necessary for the ultrasonic pulses transmitted from transducer arrays to arrive a predetermined depth position and for the reflected ultrasonic pulses to return to the source. From the calculated delay value, the average transmission velocity can be easily derived and the delay profiles along the depth direction can be calculated by using the average transmission velocity of the ultrasonic.
  • a plurality of delay profiles are applied and multiple sets of scan-line data are beam formed corresponding to each of the delay profiles. Then it is determined whether the preliminary scan-line data are accurately beam-formed, and accurately beam-formed preliminary scan-line data are synthesized into scan-line data.
  • the accuracy of the beam-forming is determined in each of determining areas, each of which is divided from the object region to be ultrasonic imaged in accordance with a predetermined criterion.
  • the accuracy of the beam-forming of the preliminary data that are generated on the basis of the delay profiles is determined separately in each of the determining areas.
  • the determining areas are further divided, or preliminary beam-forming data are synthesized. The further division of the determining areas are repeated until all of the determining areas satisfy a predetermined accuracy.
  • Fig. 2 illustrates an embodiment of the ultrasonic imaging system that beam-forms in parallel by using a plurality of delay profiles in accordance with the present invention.
  • multiple sets of preliminary scan-line data are generated by applying delay profiles in parallel.
  • the delay profiles are based on four transmission velocities (V0, V1, V2, and V4).
  • Scan-line synthesizer 240 generates adaptive scan-line data by synthesizing the multiple sets of preliminary scan-line data.
  • the reflected signals received by transducer array 10 are converted by A/D converter 20 and temporarily stored in memory (FIFO) 30.
  • the converted data are temporarily stored in each memory (FIFO0, FIFO1, FIFO2, and FIFO3) 200.
  • the digital signals stored temporarily in memory 200 are applied by delay profiles stored in delay controller 210, and then beam-formed into multiple sets of preliminary scan-line data.
  • the multiple sets of preliminary scan-line data are generated in proportion to the number of delay controllers 210 and scan-line data are synthesized by combining all sets of the preliminary scan-line data. Therefore, accurate beam-forming is possible. However, since the number of applied delay profiles might increase the system complexity and decrease operating speed, these factors should be considered in deciding the number of delay profiles.
  • the four sets of preliminary scan-line data beam-formed by four delay controllers 210 are multiplexed by multiplexer 220 and subsequently stored temporarily in memory (FIFO) 230.
  • the structure of the multiple sets of preliminary scan-line data beam-formed in parallel and shared in memory (FIFO) 230 is shown in Fig. 3.
  • the appendix (i) denotes that delay profile (i) is used. Because the preliminary scan-line data are beam-formed in parallel, all of the beam-formed data generated by four delay profiles are simultaneously obtained.
  • the method as to how scan-line synthesizer 240 generates adaptive scan-line data will be discussed in detail below.
  • the beam-former beam-forms the multiple sets of preliminary scan-line data in parallel, so that the frame rate of this embodiment is same as that of using a single delay profile.
  • the number of delay profiles increases, beam-forming channels increase so that the beam-former becomes more complicated.
  • this embodiment can be embodied onto one application-specified integrated circuit (ASIC).
  • Fig. 4 illustrates another embodiment of an ultrasonic imaging system that performs the beam-forming by applying a plurality of delay profiles.
  • the embodiment of Fig. 4 uses a single channel for beam-forming preliminary scan-line data.
  • multiple sets of preliminary scan-line data are generated by applying a plurality of delay profiles sequentially, and scan-line synthesizer 430 generates adaptive scan-line data by synthesizing all of the preliminary scan-line data.
  • transducer array 10, A/D converter 20, and memory (FIFO) 30 are same as those of Fig. 2.
  • the digital data stored temporarily in memory 30 are read.
  • controller 400 beam-forms the data read from memory 30 by applying a first delay profile.
  • the first beam-formed scan-line data are stored in memory (FIFO) 420.
  • the digital data stored temporarily in memory 30 are read and delay controller 400 beam-forms the read data by applying a second delay profile.
  • the second beam-formed scan-line data are also stored in memory (FIFO) 420.
  • all of the preliminary scan-line data are stored in memory (FIFO) 420.
  • Fig. 5 represents a structure of preliminary scan-line data that are sequentially beam-formed, according to the embodiment of the present invention of Fig. 4. Compared with the embodiment of Fig. 2, this embodiment takes four times as much as the apparatus of Fig. 3 in beam-forming four sets of preliminary scan-line data corresponding to four delay profiles.
  • delay profile storing device 410 provides delay profile that is used in delay controller 400.
  • Delay profile storing device 410 stores information on all of the delay profiles.
  • delay profile storing device 410 Upon receiving the request from delay controller 400, delay profile storing device 410 provides the requested delay profile to delay controller 400.
  • Scan-line synthesizer 430 combines multiple sets of preliminary scan-line data that are sequentially beam-formed and stored in memory (FIFO) 420 and generates adaptive scan-line data.
  • the adaptive scan-line data are synthesized by using the same method of figure 2. The detailed synthesizing method will be explained later.
  • the beam-former uses single channel, which is different from the embodiment of Fig. 2.
  • hardware complexity decreases, but the delay for generating preliminary scan-line data is proportional to the number of applied delay profiles.
  • scan-line synthesizers 230, 430 generate adaptive scan-line data by combining multiple sets of scan-line data.
  • synthesizing method should reflect the medium characteristics. For still images, it is possible to repeat the same method if the method of synthesizing scan-line data is determined by scanning the full image once. For moving images, different synthesizing methods should be adopted each time of scanning the ultrasonic images.
  • the accuracy of the beam-forming should be determined.
  • the determination of the beam-forming accuracy is performed separately in each of the areas.
  • Each area comprises a predetermined number of sets of preliminary scan-line data, which is called hereinafter "determining areas.” That is, the object region to be ultrasonic imaged is divided into determining areas, and the most accurate preliminary scan-line data are selected separately in each of the determining areas.
  • the method of generating the adaptive scan-line data will be discussed in detail with reference to Figs. 6 to 10, which is implemented in scan-line synthesizer 240, 430.
  • Fig. 6 illustrates a structure of the scan-line synthesizers 240, 430 of Figs. 2 and 4.
  • Scan-line synthesizers 240, 430 generate adaptive scan-line data by using multiple sets of beam-formed scan-line data (the data beam-formed in parallel in Fig. 2 and the data beam-formed sequentially in Fig. 4).
  • scan-line synthesizers 240, 430 determine the accuracy of the multiple sets of preliminary scan-line data, select the most accurately beam-formed preliminary scan-line data in each of the determining areas, and combine the selected preliminary scan-line data into scan-line data.
  • scan-line accuracy determining means 600 determines the accuracy of the beam-forming. As explained above, accuracy is determined separately in each of the determining areas. Regarding the determining areas, when the decision region is set up broadly, the process and the system for forming the adaptive scan-line may be simplified. Then, it becomes difficult to form accurate adaptive scan-line data. To the contrary, when the decision region is set up narrowly, the reverse results would happen. From the foregoing, it is necessary to set up compromising conditions between the accurate beam-forming of the adaptive scan-line data and the simplicity of the system.
  • area dividing means 610 performs necessary division of determining area.
  • Scan-line accuracy determining means 600 and area dividing means 610 exchange information on the accuracy and the division of determining areas each other until the determined accuracy exceeds a predetermined threshold value by dividing the area.
  • Scan-line combining means 620 generates the adaptive scan-line data on the basis of the accuracy of preliminary scan-line data provided by accuracy determining means 600 and the determining areas provided by area dividing means 610. The detailed descriptions of each of the means are as follows.
  • the method of determining the accuracy of the beam-forming by scan-line accuracy determining means 600 is explained in detail.
  • the beam-forming is accurate, the ultrasonic image will be displayed clean. Accordingly, the brightness components and the high-frequency components of pixels can be used as the criteria of the accuracy of the beam-forming.
  • the brightness components or the high frequency components are detected over the determining areas comprising the predetermined number of scan-lines.
  • the brightness components are used as the criteria of determining the accuracy of the beam-forming, the brightness components of pixels are extracted, and the average value of the brightness components is calculated. That is, the beam-forming is determined to be accurate if the average value of the brightness components over the determining areas exceeds a predetermined threshold value.
  • B(x, y) denotes the brightness component of pixel located in coordinates (x, y). Coordinates (x, y) denote one of points within the determining areas, from which the brightness components are extracted. N denotes the number of pixels distributed in the determining areas.
  • the high frequency components can be used as the criteria of determining the beam-forming accuracy.
  • the accuracy of the beam-forming can be determined by extracting the high frequency components.
  • Fig. 7 illustrates an embodiment of an apparatus for extracting the high frequency components from the scan-line data and determining the accuracy of the beam-forming.
  • Preprocessor 700 performs preprocessing function, such as automatic gain control (AGC).
  • AGC automatic gain control
  • Noise filter 710 filters out noise components. Median filter may be used as noise filter 710. Because noise components also contain high frequency components, the noise filtering is required for the noise components not to interfere the high frequency components representing contrast in the scan-line data.
  • the noise-filtered signals are inputted to high frequency extractor 720.
  • High frequency extractor 720 extracts the frequency components that belong to the desired band. In fact, high frequency extractor 720 can be composed of band-pass filters that are fit to the desired bandwidth.
  • High frequency evaluator 730 calculates an accumulated value of predetermined frequency components. When the value calculated by high frequency evaluator 730 exceeds a predetermined threshold value, it is determined that the beam-forming is accurate.
  • the average value calculated by high frequency evaluator 730 is expressed in the following formula.
  • F(x, y) denotes a predetermined frequency component
  • S denotes the size of a region where the predetermined frequency component is extract. Coordinates (x, y) are within the region where the predetermined frequency components are to be extracted.
  • area dividing means 610 divides determining areas.
  • Scan-line combining means 620 combines multiple sets of preliminary scan-line data that maximize the value of formula 1 or 2 in each of the determining area and generates adaptive scan-line data.
  • the method of hierarchical division and edge detection can be used.
  • Fig. 8 shows a method of dividing the determining area 800 two times by the hierarchical process. For convenient descriptions, it is assumed that four preliminary beam-forming data are generated by using four delay profiles.
  • the accuracy of the beam-forming of determining area 800 comprising a number of sets of scan-line data is determined by the formula 1 or 2 that calculates an accuracy of beam-forming by extracting brightness components of pixels or a certain range of frequency components.
  • multiple sets of preliminary scan-line data are beam-formed by applying four delay profiles. If the value of formula 1 or 2 in connection with one or more delay profiles exceeds a predetermined threshold, the area is not divided. In this case, the preliminary scan-line data that maximize the formula 1 or 2 are determined to be adaptive scan-line data. In Fig. 8, the preliminary scan-line data that are beam-formed by applying a delay profile generated on the basis of propagation velocity V0 are determined to be scan-line data.
  • determining area 800 is divided into first area 810 and second area 820. Though various dividing methods can be used, determining area 800 is repeatedly divided vertically or horizontally at the midpoint of a vertical and horizontal direction as shown in Fig. 8. However, it should be noted that other dividing methods could be used in other embodiments of the present invention, for example repeating one of vertical or horizontal division, or diagonal division.
  • each of the first and second parts 810, 820 that are generated by the first vertical division it is determined whether the values of formula 1 or 2 of all delay profiles exceed a predetermined threshold. If the values in connection with several delay profiles exceed the threshold, the preliminary scan-line data that maximize the value of formula 1 or 2 are determined to be scan-line data of the two divided parts 810, 820.
  • Fig. 8 shows that the preliminary scan-line data that are applied by the delay profile based upon the propagation velocity V0 are determined to be the scan-line data of the divided part 810, and the preliminary scan-line data that are applied by the delay profile based upon the propagation velocity V1 are determined to be the scan-line data of the divided part 820.
  • a second division is performed.
  • the second part 820 formed by the first division is further divided by the second division, but the first part 810 is not further divided.
  • the same processes that were performed after the first division are repeated in each of further divided parts 821, 823.
  • delay profiles based upon propagation velocity V1, V2 are selected respectively in the further division parts 821, 823.
  • Fig. 8 also shows another type of division that might be formed by the second division.
  • each of the dividing parts 810, 820 formed by the first division is further horizontally divided.
  • determining area 800 is finally divided into four division parts 811, 813, 821, 823.
  • the scan-line data of the first division part 811 are determined to be the preliminary scan-line data based upon the propagation velocity V0
  • the scan-line data of the second division part 813 are determined to be the preliminary scan-line data based upon the propagation velocity V1
  • the scan-line data of the third division part 821 are determined to be the preliminary scan-line data based upon the propagation velocity V2
  • the scan-line data of the fourth division part 823 are determined to be the preliminary scan-line data based upon the propagation velocity V3.
  • Fig. 9 shows the dividing method by means of edge detection. As shown in Fig. 9, determining area 900 is composed of materials 910 ⁇ 940 that are unknown to outside. As the case of Fig. 8, four delay profiles are assumed to be used.
  • the dividing method by means of edge detection is as follows.
  • the accuracy of the beam-forming of the object area 900 is determined by the formula 1 or 2 that determines an accuracy of the beam-forming by extracting brightness components of pixels or a certain range of frequency components. If the value of formula 1 or 2 in regard of the entire determining area exceeds a predetermined threshold, determining area is not divided. Then, one of four preliminary scan-line data that maximize formula 1 or 2 is determined to be scan-line data of the determining area 900.
  • area division is performed by detecting the edge of the materials.
  • Gaussian filter a kind of low pass filter, is used.
  • the bandwidth of the filter is determined by its variance.
  • the determining area 900 is divided into three parts 920, 940, 950 after the first edge detection. As shown in Fig. 9, the edge between the materials 910, 930 is not detected. Regarding each of the divided parts, it is determined whether the value of formula 1 or 2 exceeds a predetermined threshold. Among delay profiles that exceed the threshold, the delay profile maximizing the value of formula 1 or 2 is selected for each of the dividing parts.
  • Each of the three sets of preliminary scan-line data which are based upon the propagation velocity V1, V2, V3 respectively, are determined to be the scan-line data of each of the dividing parts 920, 940, 950.
  • the bandwidth of the median filter may be adjusted for the accurate edge detection.
  • the dividing process by means of the hierarchical dividing method or the edge detection is repeated until the value of formula 1 or 2 exceeds a threshold.
  • the number of division may be limited. In this case, at the final state of division, whether the value of formula 1 or 2 exceeds a threshold is not determined, but the preliminary scan-line data maximizing each of the dividing parts as scan-line data is determined.
  • Fig. 10 is a flow chart showing the process of synthesizing adaptive scan-line data by applying a plurality of delay profiles.
  • multiple sets of preliminary scan-line data are generated by applying a plurality of delay profiles.
  • the value of formula 1 or 2 in connection with the multiple sets of preliminary scan-line data is calculated separately in each of determining or dividing areas, and the calculated value is compared with a threshold value to determine the accuracy of the beam-forming. If the beam-forming is determined to be accurate, transit to step 1030 where adaptive scan-line data are generated by combining each of the preliminary scan-line data that maximize the value of formula 1 or 2 in each of the dividing parts.
  • the adaptive scan-line data can reflect the characteristics of the object region.
  • accurate delay profiles might be generated on the basis of the propagation velocities and refractive index in each medium. For example, in case of diagnosing an abdominal region, the object region is changed from fat to muscle along the vertical direction. In case of diagnosing a carotid, the materials that extend along the horizontal direction are composed of blood.
  • the method of generating delay profiles on the basis of the medium characteristic is as follows.
  • the object region to be ultrasonic imaged is composed of three parts. Even though the compositions of the object region are known, the thickness and refractive index of each part that ultrasonic pulses actually propagates might be different depending upon patients and operators of the ultrasonic imaging system. Therefore, delay profiles applied to the ultrasonic imaging system are generated as follows as shown in figure 2.
  • propagation velocities are estimated to be V1a ⁇ V1d in the first part of the object region, V2a ⁇ V2d in the second part of the object region, and V3a ⁇ V3d in the third part of the object region.
  • delay profile Di is generated on the basis of propagation velocity V1i, V2i, V3i("i" represents one of a ⁇ d).
  • the method for dividing each part of the object region and synthesizing preliminary scan-line data is same as the method shown in figures 2 to 9.
  • delay profiles generated on the basis of the characteristics of the object region can be generated by the different method.
  • the ultrasonic pulses go through four kinds of paths, each of which has different propagation velocity variations, namely V3, V1 ⁇ V3, V1 ⁇ V3 ⁇ V2 and V3 ⁇ V2. Therefore, the delay profiles applied in figure 2 are D1 that is based on the propagation velocity V3, D2 that is based on the propagation velocity V1 ⁇ V3, D3 that is based on the propagation velocity V1 ⁇ V3 ⁇ V2, and D4 that is based on the propagation velocity V3 ⁇ V2.
  • the object region is divided before adaptive delay profiles are applied.
  • an ultrasonic image is formed by applying a single delay profile, and then, the object region is divided.
  • Edge detection or artificial intelligence may be used in dividing the object region.
  • adaptive delay profiles are generated by one of the methods described above.
  • the adaptive transmit-focusing is different from the adaptive beam-forming in that it is impossible to perform dynamic focusing once a group of ultrasonic pulses leave transducer arrays.
  • it is needed to either generate a plurality of delay profiles on the basis of variations of a propagation velocity according to the medium variations, or transmit-focus on the basis of multiple points per scan-line. Therefore, it is necessary to either repeat transmission of transmit-focused pulses by applying plurality of delay profiles (a kind of multiplexing transmit-focusing in time-domain) or to multiplex pulses, each of which is generated on the basis of each of the plurality of delay profiles or on the basis of the multiple points of the transmit-focusing, for transmitting transmit-focused pulses at a time.
  • the transmit-focusing apparatus shown in figure 1a can be used.
  • Several propagation delays are assumed within the object region.
  • the delay profiles are generated based upon the propagation delays and applied to the delay circuit 1 of figure 1a. Since it is difficult to estimate the compositions of the object region, several delay profiles are generated from the several propagation delays assumed within the object region and applied to the delay circuit 1 of a transmit-focusing device.
  • the reflected signals from a group of ultrasonic pulses which are sequentially transmit-focused on the basis of the assumed several propagation velocities, are beam-formed to be preliminary scan-line data.
  • Preliminary scan-line data can be beam-formed by the conventional apparatus as shown in figure 1b.
  • a delay profile generated on the basis of the assumed propagation velocity is applied to the delay controller 40 of figure 1b.
  • multiple sets of preliminary scan-line data are sequentially generated on the basis of the assumed propagation velocities.
  • the multiple sets of preliminary scan-line data are divided and synthesized to be adaptive scan-line data.
  • the method shown in figures 6 to 9 can be used.
  • the conventional transmit-focusing and beam-forming apparatus can be used.
  • this method reduces frame rate in proportion to the repeating number of transmit-focusing. For example, three velocities are assumed within the object region, the frame rate of an ultrasonic image is reduced to 1/3 of the conventional image.
  • statistical information about propagation velocities within the object region is used to estimate the propagation velocity according to the direction of transmit-focusing.
  • the propagation velocities that can be induced from the delay profiles applied in the adaptive beam-forming apparatus of figures 2 and 4 can be used as the statistical information.
  • the propagation velocity of the transmitted pulses from the transducer array 40 is equal to that of the reflected pulses that return to the transducer array 40.
  • statistical information about the propagation velocities in each of diagnosing parts can be calculated in advance, and the transmit-focusing can be performed on the basis of this calculated information.
  • Figure 11 shows an embodiment of an adaptive transmit-focusing apparatus that applies a plurality of delay profiles but does not reduce frame rates.
  • an ultrasonic pulse that is transmitted via each of the transducers is generated by a plurality of delay circuits 1, a plurality of waveform tables 3, multiplexer 7 that combines input pulses into a single pulse and amplifier 5 that amplifies the pulse from multiplexer 7.
  • This apparatus compared with the conventional apparatus of figure 1a, has the difference in that it comprises a plurality of delay circuits 1a ⁇ 1c and waveform tables for generating a single pulse. Even though three delay circuits 1a ⁇ 1c and waveform tables 3a ⁇ 3c are connected to each of the transducer elements, the delay circuit 1 and the waveform table 3 can be used as needed.
  • the embodiment of figure 11 is suitable for the environments where the propagation velocity within the object region to be ultrasonic imaged varies in V1 ⁇ V3 along the direction of transmit-focusing.
  • Three delay circuits 1a ⁇ 1c and three waveform tables 3a ⁇ 3c are tuned to the propagation velocities V1 ⁇ V3.
  • Waveform data are read from each of the waveform tables 3a ⁇ 3c according to the signals from each of the delay circuits 1a ⁇ 1c and three pulses, each of which is corresponding to each of delay profiles applied to the delay circuit 1, are generated. Since the delay profiles applied to the delay circuits 1a ⁇ 1c are based upon three different propagation velocities, they usually have different values each other.
  • waveforms stored in each of the waveform tables 3a ⁇ 3c should have predetermined patterns that enable the combined signal to be separated into three signals.
  • Each of the three signals corresponds to the delay profiles applied in transmit-focusing, when the reflected signal is beam-formed.
  • frequency division multiplexing can be used.
  • waveform data stored in each of the waveform tables 3a ⁇ 3c use different carrier frequency, and the multiplexer 7 multiplexes waveform signals from three waveform tables 3a - 3c into a single frequency multiplexed pulse. Therefore, if the reflected ultrasonic signals are band-pass filtered, it is possible to demultiplex reflected ultrasonic signals into three separate signals, each of which corresponds to the delay profiles applied in transmit-focusing.
  • Code division multiplexing can be also used.
  • waveform data stored in each of waveform tables 3a ⁇ 3c are modulated by codes, for example, Golay code showing orthogonal property.
  • the multiplexer 7 multiplexes waveform signals from three waveform tables 3a ⁇ 3c into a single code-division multiplexed pulse. Therefore, if the reflected ultrasonic signals pass filters that are matched with the codes, it is possible to demultiplex reflected ultrasonic signals into three separate signals, each of which corresponds to the delay profiles applied in transmit-focusing.
  • Figure 12 shows one embodiment of a receiving apparatus that is able to demultiplex the reflected ultrasonic signals into separated signals in case that the ultrasonic signals are transmit-focused by the transmitting apparatus of figure 11.
  • signals received via a receiving part of transducer arrays 10 are beam-formed by the conventional beam-forming apparatus.
  • the beam-formed data pass three filters 1230a ⁇ 1230c, the beam-formed data are demultiplexed into three separated signals, each of which corresponds to each of delay profiles applied in the transmit-focusing.
  • the filter 1230 is band pass filter where the center frequency is matched to the carrier frequency of waveform data stored in the waveform table 3.
  • code division multiplexing is used, the filter 1230 is matched with codes used in waveform data stored in the waveform table 3.
  • Dividing and synthesizing circuit 1240 divides the multiple sets of preliminary scan-line data and synthesizes the divided scan-line data to form an adaptive scan-line data.
  • the dividing and synthesizing method of figures 6 to 9 can be applied in the dividing and synthesizing circuit 1240.
  • Figure 13 shows another embodiment of a receiving apparatus that is able to demultiplex the reflected ultrasonic signals into separated signals when the ultrasonic signals are transmit-focused by the transmitting apparatus of figure 11.
  • this embodiment has the difference in that filters 1230a ⁇ 1230c demultiplexing the reflected signals are connected between transducer array 10 and beam-former 1210.
  • This embodiment demultiplexes received signals into separate signals more accurately than the apparatus of figure 12 because the received signals are demultiplexed before they are beam-formed.
  • Each of the separated signals corresponds to each of delay profiles applied in the transmit-focusing.
  • demultiplexing is performed in each of the transducer elements and, therefore, it becomes more complicated than the embodiment of figure 12.
  • dividing and synthesizing circuit 1240 divides the multiple sets of preliminary scan-line data and synthesizes the divided scan-line data into adaptive scan-line data.
  • the transmit-focusing method of applying a plurality of delay profiles generated on the basis of the propagation velocity in accordance with the medium is applicable to the method of transmit-focusing on multiple points per a scan-line. That is, delay profiles that are applied to the delay circuit 1 are generated in such a way that each of the delay profiles associates each of the multiple transmit-focusing points.
  • Each of the delay circuits 1 is applied by the associated delay profile that is optimized one of the multiple transmit-focusing points.
  • the waveform tables 3 store waveform data in such a way that the received signals can be separated into a plurality of preliminary scan-line data when multiple pulses, each of which is associated with each of the multiple transmit-focusing points, are combined into a signal for transmission.
  • the filters 1230 of figures 12 and 13 are matched in such a way that the received signals can be separated into these individual signals.
  • the transmit-focusing points can be located in each of the determining areas.
  • determining area 800 is divided into four dividing parts 811, 813, 832, 823, and transmit-focusing point is located at the center point in an axial direction in each of the four dividing parts 811, 813, 832, 823.
  • ultrasonic pulses are transmit-focused along the center points on the bases of propagation velocity V0.
  • ultrasonic pulses are transmit-focused along the center points on the bases of propagation velocity V1 in dividing part 811, ultrasonic pulses are transmit-focused along the center points on the bases of propagation velocity V2, and in dividing part 823, ultrasonic pulses are transmit-focused along the center points on the bases of propagation velocity V3.
  • dividing part 950 since dividing part 950 is concaved by dividing part 920, the upper and lower parts of dividing part 950 are respectively transmit-focused.
  • ultrasonic pulses are transmit-focused along the center points of each the upper and lower parts of dividing part 950 on the basis of propagation velocity V2.
  • ultrasonic pulses are transmit-focused along the center points of dividing part 920 on the basis of propagation velocity V1.
  • Ultrasonic pulses are also transmit-focused along the center points of dividing part 940 on the basis of propagation velocity V3.
  • the aforementioned transmit-focusing and beam-forming method is applicable in a multi-beam system.
  • the transmit-focusing using adaptive delay profiles as shown in figures 11 is directly applicable in the transmitting device of the multi-beam system.
  • the receiving apparatus of figure 12, 13, which is used when the adaptive transmit-focusing is applied, is also directly applicable to each channel of the multi-beam system that generates each of the multiple beams.
  • the beam-forming using adaptive delay profiles is also directly applicable to each channel of the multi-beam system that generates each of the multiple beams.
  • the transmitting apparatus of figure 11 and the receiving apparatus of figure 12 or 13 are used in each of the transmit-focusing and beam-forming.
  • the adaptive beam-forming apparatus of figure 2 or 4 is applied in the beam-former 1210 of figure 12 or 13.
  • an adaptive transmit-focusing and beam-forming method and apparatus can reduce errors of transmit-focusing and beam-forming, which occur from the inaccuracy of applied delay profile due to medium variations.
  • the present invention provides a clear ultrasonic image because it transmit-focuses and beam-forms by applying accurate delay profiles generated on the basis of the medium characteristics.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)
  • Investigating Or Analyzing Materials By The Use Of Ultrasonic Waves (AREA)
EP02000322A 2001-01-10 2002-01-04 Procédé et dispositif de focalisation en transmission et de formation de faisceau Expired - Lifetime EP1225566B1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
KR20010001249 2001-01-10
KR2001001249 2001-01-10
KR2001088786 2001-12-31
KR10-2001-0088786A KR100441067B1 (ko) 2001-01-10 2001-12-31 적응적 주사선 데이터를 형성하는 송신집속 및 수신집속장치 및 방법

Publications (3)

Publication Number Publication Date
EP1225566A2 true EP1225566A2 (fr) 2002-07-24
EP1225566A3 EP1225566A3 (fr) 2004-08-04
EP1225566B1 EP1225566B1 (fr) 2006-04-05

Family

ID=36274093

Family Applications (1)

Application Number Title Priority Date Filing Date
EP02000322A Expired - Lifetime EP1225566B1 (fr) 2001-01-10 2002-01-04 Procédé et dispositif de focalisation en transmission et de formation de faisceau

Country Status (4)

Country Link
US (1) US6719695B2 (fr)
EP (1) EP1225566B1 (fr)
JP (1) JP3939153B2 (fr)
DE (1) DE60210352T2 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108663434A (zh) * 2018-04-26 2018-10-16 中国能源建设集团安徽电力建设第工程有限公司 一种相控阵超声探伤仪的粗晶材料全聚焦检测方法

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7508113B2 (en) * 2004-05-18 2009-03-24 Siemens Medical Solutions Usa, Inc. Apparatus for two-dimensional transducers used in three-dimensional ultrasonic imaging
EP1621879B1 (fr) * 2004-07-23 2010-02-10 Betriebsforschungsinstitut VDEh Institut für angewandte Forschung GmbH Recepteur ultrasonore avec digitalisation prématuré de signaux et son utilisation
JP2009089940A (ja) * 2007-10-10 2009-04-30 Toshiba Corp 超音波診断装置
JP5555416B2 (ja) * 2007-10-25 2014-07-23 三星メディソン株式会社 超音波診断装置及びスキャンラインデータ形成方法
JP2008259891A (ja) * 2008-08-04 2008-10-30 Hitachi Medical Corp 超音波診断装置
WO2011001867A1 (fr) * 2009-06-29 2011-01-06 株式会社 日立メディコ Dispositif de diagnostic à ondes ultrasonores et procédé de correction de vitesse sonore
JP6088165B2 (ja) * 2012-07-04 2017-03-01 古野電気株式会社 探知装置、探知方法及び探知プログラム
US10228354B2 (en) * 2012-10-26 2019-03-12 Nordson Corporation Single channel scanning acoustic microscope with multiple focused ultrasonic transducers
EP2920722B1 (fr) 2012-11-16 2021-01-20 Siemens Healthcare Diagnostics Inc. Procédé d'identification d'un traitement optimal d'une coronaropathie
US9720121B2 (en) 2015-01-28 2017-08-01 Baker Hughes Incorporated Devices and methods for downhole acoustic imaging
DE102015122197A1 (de) * 2015-05-11 2016-11-17 Ge Sensing & Inspection Technologies Gmbh Verfahren und Vorrichtung zur zerstörungsfreien Prüfung eines Prüflings mittels Ultraschall
JP6563800B2 (ja) * 2015-12-17 2019-08-21 オリンパス株式会社 超音波観測装置、超音波観測装置の作動方法および超音波観測装置の作動プログラム

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4566459A (en) * 1983-02-14 1986-01-28 Hitachi, Ltd. Ultrasonic diagnosis system
EP0421279A1 (fr) * 1989-09-29 1991-04-10 Terumo Kabushiki Kaisha Appareil diagnostique à ultrason avec distribution sélective de focalisation
US5331964A (en) * 1993-05-14 1994-07-26 Duke University Ultrasonic phased array imaging system with high speed adaptive processing using selected elements
US5551433A (en) * 1994-08-05 1996-09-03 Acuson Corporation Method and apparatus for a geometric aberration transform in an adaptive focusing ultrasound beamformer system
US5638820A (en) * 1996-06-25 1997-06-17 Siemens Medical Systems, Inc. Ultrasound system for estimating the speed of sound in body tissue
US5908391A (en) * 1997-05-07 1999-06-01 General Electric Company Method and apparatus for enhancing resolution and sensitivity in color flow ultrasound imaging using multiple transmit focal zones
US6056693A (en) * 1999-08-16 2000-05-02 General Electric Company Ultrasound imaging with synthetic transmit focusing

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5415173A (en) * 1993-02-23 1995-05-16 Hitachi, Ltd. Ultrasound diagnosis system
IT1264947B1 (it) * 1993-07-19 1996-10-17 Fin Gef Srl Motore a combustione interna a quattro tempi, di elevata semplicita' meccanica
US5570691A (en) * 1994-08-05 1996-11-05 Acuson Corporation Method and apparatus for real-time, concurrent adaptive focusing in an ultrasound beamformer imaging system
US6120450A (en) * 1995-01-23 2000-09-19 Commonwealth Scientific And Industrial Research Organisation Phase and/or amplitude aberration correction for imaging
US5673699A (en) * 1996-05-31 1997-10-07 Duke University Method and apparatus for abberation correction in the presence of a distributed aberrator

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4566459A (en) * 1983-02-14 1986-01-28 Hitachi, Ltd. Ultrasonic diagnosis system
EP0421279A1 (fr) * 1989-09-29 1991-04-10 Terumo Kabushiki Kaisha Appareil diagnostique à ultrason avec distribution sélective de focalisation
US5331964A (en) * 1993-05-14 1994-07-26 Duke University Ultrasonic phased array imaging system with high speed adaptive processing using selected elements
US5551433A (en) * 1994-08-05 1996-09-03 Acuson Corporation Method and apparatus for a geometric aberration transform in an adaptive focusing ultrasound beamformer system
US5638820A (en) * 1996-06-25 1997-06-17 Siemens Medical Systems, Inc. Ultrasound system for estimating the speed of sound in body tissue
US5908391A (en) * 1997-05-07 1999-06-01 General Electric Company Method and apparatus for enhancing resolution and sensitivity in color flow ultrasound imaging using multiple transmit focal zones
US6056693A (en) * 1999-08-16 2000-05-02 General Electric Company Ultrasound imaging with synthetic transmit focusing

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
FLAX S W ET AL: "PHASE-ABERRATION CORRECTION USING SIGNALS FROM POINT REFLECTORS AND DIFFUSE SCATTERERS: BASIC PRINCIPLES" IEEE TRANSACTIONS ON ULTRASONICS, FERROELECTRICS AND FREQUENCY CONTROL, IEEE INC. NEW.YORK, US, vol. 35, no. 6, 1 November 1988 (1988-11-01), pages 758-767, XP000032893 ISSN: 0885-3010 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108663434A (zh) * 2018-04-26 2018-10-16 中国能源建设集团安徽电力建设第工程有限公司 一种相控阵超声探伤仪的粗晶材料全聚焦检测方法

Also Published As

Publication number Publication date
US20020121142A1 (en) 2002-09-05
DE60210352T2 (de) 2007-01-11
DE60210352D1 (de) 2006-05-18
US6719695B2 (en) 2004-04-13
JP3939153B2 (ja) 2007-07-04
EP1225566B1 (fr) 2006-04-05
EP1225566A3 (fr) 2004-08-04
JP2002209887A (ja) 2002-07-30

Similar Documents

Publication Publication Date Title
US6638227B2 (en) Ultrasound imaging method and apparatus using orthogonal Golay codes
KR100406098B1 (ko) 가중된 직교 쳐프 신호를 이용한 동시 다중 송신 집속기반의 초음파 영상 형성 장치 및 방법
EP2019624B1 (fr) Émission synthétique ultrasonique se focalisant sur un formeur de faisceaux multilignes
US7542790B2 (en) Apparatus and method for velocity estimation in synthetic aperture imaging
EP1225566B1 (fr) Procédé et dispositif de focalisation en transmission et de formation de faisceau
US6547733B2 (en) Ultrasound imaging apparatus and method using Golay codes with orthogonal property
JP4795675B2 (ja) 医療用超音波システム
KR100875203B1 (ko) 초음파 영상의 획득 방법
JP2002045359A (ja) フィルター係数決定方法、超音波撮像システム及びその方法
JPH10507101A (ja) 超音波ビーム生成器結像システムにおけるリアルタイムで同時の適応式焦点調節の方法と装置
JPH10506800A (ja) 超音波画像における調節可能な周波数走査方法および装置
US6733453B2 (en) Elevation compounding for ultrasound imaging
US20100150412A1 (en) Adaptive parallel artifact mitigation
EP1126288A2 (fr) Système d'imagerie à ultrasons employant une mise au point de réception aux points qui correspondent aux pixels d'affichage et procédé correspondant
EP1042985A1 (fr) Dispositif de diagnostic ultrasonore
CA2166579A1 (fr) Systeme de diagnostic aux ultrasons pour imagerie tomographique
US20080030581A1 (en) Multizone Color Doppler Beam Transmission Method
KR100947827B1 (ko) 초음파 신호의 음속을 설정하는 방법
US20060184030A1 (en) Ultrasound diagnostic apparatus
KR100441067B1 (ko) 적응적 주사선 데이터를 형성하는 송신집속 및 수신집속장치 및 방법
JP4632517B2 (ja) 送受信波の整相方法及び超音波診断装置
JP2003190164A (ja) 超音波撮像装置及びその方法

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR

AX Request for extension of the european patent

Free format text: AL;LT;LV;MK;RO;SI

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR

AX Request for extension of the european patent

Extension state: AL LT LV MK RO SI

RIC1 Information provided on ipc code assigned before grant

Ipc: 7G 10K 11/34 B

Ipc: 7G 01S 15/89 B

Ipc: 7G 01S 7/52 A

17P Request for examination filed

Effective date: 20040930

17Q First examination report despatched

Effective date: 20050211

AKX Designation fees paid

Designated state(s): DE FR IT

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): DE FR IT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT;WARNING: LAPSES OF ITALIAN PATENTS WITH EFFECTIVE DATE BEFORE 2007 MAY HAVE OCCURRED AT ANY TIME BEFORE 2007. THE CORRECT EFFECTIVE DATE MAY BE DIFFERENT FROM THE ONE RECORDED.

Effective date: 20060405

REF Corresponds to:

Ref document number: 60210352

Country of ref document: DE

Date of ref document: 20060518

Kind code of ref document: P

ET Fr: translation filed
PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20070108

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 15

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 16

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20191209

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20191205

Year of fee payment: 19

Ref country code: IT

Payment date: 20200113

Year of fee payment: 19

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 60210352

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210803

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210104