EP2059068A1 - Method for three-dimensional representation of a hearing device on a head and corresponding graphic device - Google Patents

Method for three-dimensional representation of a hearing device on a head and corresponding graphic device Download PDF

Info

Publication number
EP2059068A1
EP2059068A1 EP08164382A EP08164382A EP2059068A1 EP 2059068 A1 EP2059068 A1 EP 2059068A1 EP 08164382 A EP08164382 A EP 08164382A EP 08164382 A EP08164382 A EP 08164382A EP 2059068 A1 EP2059068 A1 EP 2059068A1
Authority
EP
European Patent Office
Prior art keywords
head
virtual
hearing
hearing device
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP08164382A
Other languages
German (de)
French (fr)
Other versions
EP2059068B1 (en
Inventor
Andreas Kiepfer
Wee Haw Koo
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sivantos Pte Ltd
Original Assignee
Siemens Medical Instruments Pte Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Siemens Medical Instruments Pte Ltd filed Critical Siemens Medical Instruments Pte Ltd
Publication of EP2059068A1 publication Critical patent/EP2059068A1/en
Application granted granted Critical
Publication of EP2059068B1 publication Critical patent/EP2059068B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/65—Housing parts, e.g. shells, tips or moulds, or their manufacture
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2225/00—Details of deaf aids covered by H04R25/00, not provided for in any of its subgroups
    • H04R2225/77—Design aspects, e.g. CAD, of hearing aid tips, moulds or housings

Definitions

  • the present invention relates to a method for displaying a hearing device on a user's head. Moreover, the present invention relates to a corresponding graphics device for displaying a hearing device.
  • hearing device is understood to mean here any sound-emitting device which can be worn on or in the ear, in particular a hearing device, a headset, headphones and the like.
  • Hearing aids are portable hearing aids that are used to care for the hearing impaired.
  • different types of hearing aids such as behind-the-ear hearing aids (BTE), hearing aid with external receiver (RIC: receiver in the canal) and in-the-ear hearing aids (IDO), e.g. Concha hearing aids or canal hearing aids (ITE, CIC).
  • BTE behind-the-ear hearing aids
  • RIC hearing aid with external receiver
  • IDO in-the-ear hearing aids
  • ITE canal hearing aids
  • the hearing aids listed by way of example are worn on the outer ear or in the ear canal.
  • bone conduction hearing aids, implantable or vibrotactile hearing aids are also available on the market. The stimulation of the damaged hearing takes place either mechanically or electrically.
  • Hearing aids have in principle as essential components an input transducer, an amplifier and an output transducer.
  • the input transducer is usually a sound receiver, z. As a microphone, and / or an electromagnetic receiver, for. B. an induction coil.
  • the output transducer is usually used as an electroacoustic transducer, z. As miniature speaker, or as an electromechanical transducer, z. B. bone conduction, realized.
  • the amplifier is usually integrated in a signal processing unit. This basic structure is in FIG. 1 shown using the example of a behind-the-ear hearing aid. In a hearing aid housing 1 for Carrying behind the ear, one or more microphones 2 are installed for recording the sound from the environment.
  • a signal processing unit 3 which is also integrated in the hearing aid housing 1, processes the microphone signals and amplifies them.
  • the output signal of the signal processing unit 3 is transmitted to a loudspeaker or earpiece 4, which outputs an acoustic signal.
  • the sound is optionally transmitted via a sound tube, which is fixed with an earmold in the ear canal, to the eardrum of the device carrier.
  • the power supply of the hearing device and in particular of the signal processing unit 3 is carried out by a likewise integrated into the hearing aid housing 1 battery. 5
  • US 2003/0123026 A1 an adjustment system for spectacles.
  • a photograph is taken by the user.
  • a 3D model of the user's face is created.
  • a 3D representation of a pair of glasses is fitted into the 3D model of the face, and the result is displayed graphically.
  • US 2004/0165740 A1 and US 2005/0088435 A1 also disclose systems for creating 3D images of an ear canal and manufacturing processes for making earmolds from these 3D images.
  • the object of the present invention is thus to assist a potential user of a hearing device in the selection of a device with regard to aesthetic aspects.
  • this object is achieved by a method for displaying a hearing device on a user's head by providing a 3D model of the hearing device as a virtual hearing device, obtaining a 3D model of the user's head or a part thereof as a virtual head, aligning the virtual hearing device on the virtual head and graphically representing the virtual head with the fitted virtual hearing device.
  • the invention provides a graphic device for displaying a hearing device on a user's head comprising a memory device for providing a 3D model of the hearing device as a virtual hearing device, an optical device for obtaining a 3D model of the user's head or a part thereof as virtual Head, a computing device for aligning the virtual hearing device on the virtual head and a graphics device for graphically displaying the virtual head with the fitted virtual hearing device.
  • the hearing impaired a new hearing aid can be presented during the sales call optically.
  • the method according to the invention or the graphic device according to the invention it is possible to show the user a picture or a mirror image of him including a new hearing aid or to give it to the hand, because the acoustician will certainly the device specifically suitable not available as a demonstration device.
  • the virtual head is obtained from digital data of a standard header and at least one digital photograph of the user's head or portion thereof.
  • the standard head can be selected from several standard heads.
  • the computational effort to create a three-dimensional virtual head is significantly reduced.
  • the virtual head is now generated by means of a skeleton animation from the standard header.
  • This skeleton animation contributes significantly to the reduction of the computational effort.
  • the design of the virtual head determines with the aid of an image recognition the shape and / or the position of a given location of the photograph.
  • the chin and cheekbones can be detected automatically and the standard head can be automatically changed according to the photograph.
  • the virtual hearing device can be provided, for example, from a database with a plurality of such hearing devices. This can be done without any computational effort, a preselection or a final choice of the hearing.
  • the corresponding virtual hearing device is obtained by scanning the auditory canal of the user with subsequent 3D model formation.
  • an impression can be gained as to how far the hearing device can be inserted into the auditory canal.
  • FIG. 2 a situation is illustrated where a hearing impaired 10 visits an acoustician 11 to have a hearing aid fitted.
  • the acoustician 11 creates a virtual 3D image from the head of the hearing impaired 10.
  • the acoustician needs at least three 2D shots or photographs of the head.
  • the recordings commercial webcams or even digicams are sufficient.
  • To create a 3D model of the head is conveniently used a database of standard heads, from which then a lifelike image of the head of the hearing impaired 10 can be generated. It is then only necessary to select the most suitable standard head and to modify the striking parts of the head according to the photographs.
  • the head For this purpose, for example, serve an image recognition, which recognizes in the photos the ears, the eyes, the eyebrows, the lips, the nose or the cheekbones. Based on this recognition, the corresponding sections of the standard head can then be changed in shape and position. For this purpose, it is again favorable to use the known skeletal animation technology, since changes can be carried out with less computational effort on the basis of simple skeleton or skeleton representations. Based on this, the final 3D model of the head can then be synthesized, which is also possible with well-known technologies (morphing).
  • a 3D model of the hearing device to be used is available, for example, for BTE hearing aids in the form of CAD data in a database.
  • scan data or the data of prefabricated IdO shells can serve as the basis for the respective 3D model.
  • FIG. 3 a section of the image 13 is shown enlarged. It thus shows not only the virtual head 14 ', but also the virtual hearing aid 15'. Both, both the head 14 'and the hearing aid 15' are present as a three-dimensional data set, so that the image 13 can be rotated accordingly. Thus, the hearing impaired 10 can view his head 14 'with the attached hearing aid 15' virtually from different perspectives. It is easily possible to change and present the type, the color and the size of the virtual hearing device 15.
  • the 4 to 7 show a somewhat simplified embodiment of the present invention. According to this embodiment, only an ear portion 16 of the hearing impaired 10 is photographed for forming the 3D model of a part of the head. As used herein, this header is also referred to as a "virtual head". In order to obtain the image, for example, a webcam or video camera 17 is installed on the screen 18 of a PC, which serves as an adapter.
  • FIG. 5 now shows an image that can be displayed on the screen 18 using a fitting software (FSW: Fitting Software).
  • FSW Fitting Software
  • an image section 19 is a 3D video as a virtual head section 16 'with a virtual hearing aid 20 'shown. It can also be seen there how the virtual receiver tube 21 'of the virtual BTE hearing device 20' is guided around the virtual auricle 22 'and opens into a virtual earmold 23' in the center of the virtual auricle 22 '.
  • FIG. 5 the virtual head section 16 'is shown from the side. But it can also according to FIG. 6 a sight of the virtual ear portion 16 'are presented from the front. Likewise, according to FIG. 7 represent how the hearing aid 20 'looks in the worn state from behind.
  • the virtual representations can either be as single shots as in the FIGS. 5 to 7 or presented as part of a moving video film. It is - as mentioned - possible to change the model or the color of the hearing aid shown. In particular, thus also BTE, ITE or CIC hearing aids can be presented virtually. Based on these three-dimensional representations, the hearing-impaired person can then better decide which type of hearing aid or which color he chooses. This increases the satisfaction when wearing the hearing aid, at least in terms of appearance.

Landscapes

  • Engineering & Computer Science (AREA)
  • Manufacturing & Machinery (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Neurosurgery (AREA)
  • Otolaryngology (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The method involves providing a three dimensional model of a hearing device as a virtual hearing device (20'). A three dimensional model of the head of a user or a part of it is obtained as a virtual head. The virtual hearing device is adjusted at the virtual head. The virtual head is graphically represented with the adjusted virtual hearing device. The virtual head is obtained from digital data of a standard head and a digital photography of the head of the user. An independent claim is included for a drawing device for representation of a hearing device at a head of a user.

Description

Die vorliegende Erfindung betrifft ein Verfahren zur Darstellung einer Hörvorrichtung an einem Kopf eines Nutzers. Darüber hinaus betrifft die vorliegende Erfindung eine entsprechende Grafikvorrichtung zur Darstellung einer Hörvorrichtung. Unter dem Begriff "Hörvorrichtung" wird hier jedes am oder im Ohr tragbare, schallausgebende Gerät, insbesondere ein Hörgerät, ein Headset, Kopfhörer und dergleichen, verstanden.The present invention relates to a method for displaying a hearing device on a user's head. Moreover, the present invention relates to a corresponding graphics device for displaying a hearing device. The term "hearing device" is understood to mean here any sound-emitting device which can be worn on or in the ear, in particular a hearing device, a headset, headphones and the like.

Hörgeräte sind tragbare Hörvorrichtungen, die zur Versorgung von Schwerhörenden dienen. Um den zahlreichen individuellen Bedürfnissen entgegenzukommen, werden unterschiedliche Bauformen von Hörgeräten wie Hinter-dem-Ohr-Hörgeräte (HdO), Hörgerät mit externem Hörer (RIC: receiver in the canal) und In-dem-Ohr-Hörgeräte (IdO), z.B. auch Concha-Hörgeräte oder Kanal-Hörgeräte (ITE, CIC), bereitgestellt. Die beispielhaft aufgeführten Hörgeräte werden am Außenohr oder im Gehörgang getragen. Darüber hinaus stehen auf dem Markt aber auch Knochenleitungshörhilfen, implantierbare oder vibrotaktile Hörhilfen zur Verfügung. Dabei erfolgt die Stimulation des geschädigten Gehörs entweder mechanisch oder elektrisch.Hearing aids are portable hearing aids that are used to care for the hearing impaired. In order to meet the numerous individual needs, different types of hearing aids such as behind-the-ear hearing aids (BTE), hearing aid with external receiver (RIC: receiver in the canal) and in-the-ear hearing aids (IDO), e.g. Concha hearing aids or canal hearing aids (ITE, CIC). The hearing aids listed by way of example are worn on the outer ear or in the ear canal. In addition, bone conduction hearing aids, implantable or vibrotactile hearing aids are also available on the market. The stimulation of the damaged hearing takes place either mechanically or electrically.

Hörgeräte besitzen prinzipiell als wesentliche Komponenten einen Eingangswandler, einen Verstärker und einen Ausgangswandler. Der Eingangswandler ist in der Regel ein Schallempfänger, z. B. ein Mikrofon, und/oder ein elektromagnetischer Empfänger, z. B. eine Induktionsspule. Der Ausgangswandler ist meist als elektroakustischer Wandler, z. B. Miniaturlautsprecher, oder als elektromechanischer Wandler, z. B. Knochenleitungshörer, realisiert. Der Verstärker ist üblicherweise in eine Signalverarbeitungseinheit integriert. Dieser prinzipielle Aufbau ist in FIG 1 am Beispiel eines Hinter-dem-Ohr-Hörgeräts dargestellt. In ein Hörgerätegehäuse 1 zum Tragen hinter dem Ohr sind ein oder mehrere Mikrofone 2 zur Aufnahme des Schalls aus der Umgebung eingebaut. Eine Signalverarbeitungseinheit 3, die ebenfalls in das Hörgerätegehäuse 1 integriert ist, verarbeitet die Mikrofonsignale und verstärkt sie. Das Ausgangssignal der Signalverarbeitungseinheit 3 wird an einen Lautsprecher bzw. Hörer 4 übertragen, der ein akustisches Signal ausgibt. Der Schall wird gegebenenfalls über einen Schallschlauch, der mit einer Otoplastik im Gehörgang fixiert ist, zum Trommelfell des Geräteträgers übertragen. Die Stromversorgung des Hörgeräts und insbesondere die der Signalverarbeitungseinheit 3 erfolgt durch eine ebenfalls ins Hörgerätegehäuse 1 integrierte Batterie 5.Hearing aids have in principle as essential components an input transducer, an amplifier and an output transducer. The input transducer is usually a sound receiver, z. As a microphone, and / or an electromagnetic receiver, for. B. an induction coil. The output transducer is usually used as an electroacoustic transducer, z. As miniature speaker, or as an electromechanical transducer, z. B. bone conduction, realized. The amplifier is usually integrated in a signal processing unit. This basic structure is in FIG. 1 shown using the example of a behind-the-ear hearing aid. In a hearing aid housing 1 for Carrying behind the ear, one or more microphones 2 are installed for recording the sound from the environment. A signal processing unit 3, which is also integrated in the hearing aid housing 1, processes the microphone signals and amplifies them. The output signal of the signal processing unit 3 is transmitted to a loudspeaker or earpiece 4, which outputs an acoustic signal. The sound is optionally transmitted via a sound tube, which is fixed with an earmold in the ear canal, to the eardrum of the device carrier. The power supply of the hearing device and in particular of the signal processing unit 3 is carried out by a likewise integrated into the hearing aid housing 1 battery. 5

Bei der Auswahl und der Anpassung von Hörgeräten spielen subjektive, ästhetische Eindrücke eine wesentliche Rolle. Vielfach verfügen Akustiker daher über Demonstrationsgeräte, die der Hörgeschädigte probeweise anlegen kann. Dies gilt insbesondere für HdO-Geräte, welche hinter das Ohr gehängt werden können. In der Regel ist aber der Fall auszuschließen, dass der Akustiker ein Demonstrationsgerät mit genau der gewünschten Farbe, der gewünschten Form eines geeigneten Ohrpassstücks usw. anbieten kann. Bei IdO-Geräten ist das Problem noch größer, denn der Nutzer wird kein Demonstrationsgerät zur Verfügung gestellt bekommen können, das in seinen Gehörgang passt und die erforderliche Leistung bringt. Daher wird sich der Nutzer ein neues Gerät nicht an seinem Kopf vorstellen können.In the selection and adaptation of hearing aids subjective, aesthetic impressions play an essential role. In many cases, therefore, acousticians have demonstration devices that the hearing-impaired can test-put on. This is especially true for BTE devices that can be hung behind the ear. In general, however, the case is ruled out that the acoustician can offer a demonstration device with exactly the desired color, the desired shape of a suitable ear tip, etc. For IdO devices, the problem is even greater because the user will not be able to get a demonstration device that fits into his ear canal and provides the required performance. Therefore, the user will not be able to imagine a new device on his head.

Zur optischen Demonstration eines Hörgeräts im getragenen Zustand ist man bislang auf eine Anpasssoftware angewiesen, bei der ein Foto einer Dame zu sehen ist, die ein vorgegebenes Hörgerät in einer Standardfarbe trägt. Dieses Bild soll dem Hörgeschädigten helfen, sich das Hörgerät an sich selbst vorzustellen. Dies gelingt jedoch nur in wenigen Fällen.For the optical demonstration of a hearing aid in the worn state, it has hitherto been dependent on fitting software in which a photo of a lady wearing a given hearing aid in a standard color can be seen. This image is to help the hearing impaired to imagine the hearing aid to themselves. However, this succeeds only in a few cases.

Aus der Druckschrift EP 1 246 506 A1 ist ein CAD/CAM-System zum Entwerfen eines Hörgeräts bekannt. Von dem Gehörgang eines Nutzers wird die Kontur bestimmt und daraus ein 3D- Computermodell erzeugt. Mit Hilfe einer oder mehrerer Datenbanken wird das Modell vervollständigt. Schließlich wird aus dem vervollständigten Modell eine Hörgeräteschale gefertigt.From the publication EP 1 246 506 A1 For example, a CAD / CAM system for designing a hearing aid is known. From the auditory canal of a user, the contour is determined and from it a 3D computer model generated. With the help of one or more databases, the model is completed. Finally, a hearing aid shell is made from the completed model.

Weiterhin offenbart die Druckschrift US 2003/0123026 A1 ein Anpasssystem für Brillen. Von dem Nutzer wird eine photographische Aufnahme genommen. Mit Hilfe eines Datenbankarchivs wird daraus dann ein 3D-Modell des Gesichts des Nutzers erstellt. Eine 3D-Repräsentation einer Brille wird in das 3D-Modell des Gesichts eingepasst, und das Ergebnis wird grafisch dargestellt.Furthermore, the document discloses US 2003/0123026 A1 an adjustment system for spectacles. A photograph is taken by the user. With the help of a database archive, a 3D model of the user's face is created. A 3D representation of a pair of glasses is fitted into the 3D model of the face, and the result is displayed graphically.

Die weiteren Druckschriften US 5 056 204 , US 2004/0165740 A1 und US 2005/0088435 A1 offenbaren ebenfalls Systeme zum Erstellen von 3D-Abbildungen eines Gehörgangs sowie Herstellungsverfahren zum Herstellen von Otoplastiken aus diesen 3D-Bildern.The other pamphlets US 5 056 204 . US 2004/0165740 A1 and US 2005/0088435 A1 also disclose systems for creating 3D images of an ear canal and manufacturing processes for making earmolds from these 3D images.

Die Aufgabe der vorliegenden Erfindung besteht somit darin, einen potentiellen Nutzer einer Hörvorrichtung bei der Auswahl eines Geräts hinsichtlich ästhetischer Aspekte zu unterstützen.The object of the present invention is thus to assist a potential user of a hearing device in the selection of a device with regard to aesthetic aspects.

Erfindungsgemäß wird diese Aufgabe gelöst durch ein Verfahren zur Darstellung einer Hörvorrichtung an einem Kopf eines Nutzers durch Bereitstellen eines 3D-Modells der Hörvorrichtung als virtuelle Hörvorrichtung, Gewinnen eines 3D-Modells des Kopfs des Nutzers oder eines Teils davon als virtuellen Kopf, Ausrichten der virtuellen Hörvorrichtung an dem virtuellen Kopf und grafisches Darstellen des virtuellen Kopfs mit der eingepassten virtuellen Hörvorrichtung.According to the invention, this object is achieved by a method for displaying a hearing device on a user's head by providing a 3D model of the hearing device as a virtual hearing device, obtaining a 3D model of the user's head or a part thereof as a virtual head, aligning the virtual hearing device on the virtual head and graphically representing the virtual head with the fitted virtual hearing device.

Darüber hinaus wird erfindungsgemäß bereitgestellt eine Grafikvorrichtung zur Darstellung einer Hörvorrichtung an einem Kopf eines Nutzers umfassend eine Speichereinrichtung zum Bereitstellen eines 3D-Modells der Hörvorrichtung als virtuelle Hörvorrichtung, eine Optikeinrichtung zum Gewinnen eines 3D-Modells des Kopfs des Nutzers oder eines Teils davon als virtuellen Kopf, eine Recheneinrichtung zum Ausrichten der virtuellen Hörvorrichtung an dem virtuellen Kopf und eine Grafikeinrichtung zum grafischen Darstellen des virtuellen Kopfs mit der eingepassten virtuellen Hörvorrichtung.In addition, the invention provides a graphic device for displaying a hearing device on a user's head comprising a memory device for providing a 3D model of the hearing device as a virtual hearing device, an optical device for obtaining a 3D model of the user's head or a part thereof as virtual Head, a computing device for aligning the virtual hearing device on the virtual head and a graphics device for graphically displaying the virtual head with the fitted virtual hearing device.

In vorteilhafter Weise ist es so möglich, ein noch nicht hergestelltes, virtuelles Hörgerät oder eine entsprechende andere Hörvorrichtung direkt am "Nutzer" zu präsentieren. Insbesondere kann so beispielsweise dem Hörgeschädigten ein neues Hörgerät während des Verkaufsgesprächs optisch dargeboten werden. Mit anderen Worten, mit dem erfindungsgemäßen Verfahren bzw. der erfindungsgemäßen Grafikvorrichtung ist es möglich, dem Nutzer ein Bild bzw. ein Spiegelbild von ihm inklusive einem neuen Hörgerät zu zeigen bzw. an die Hand zu geben, denn der Akustiker wird das konkret passende Gerät sicherlich nicht als Demonstrationsgerät zur Verfügung haben.In an advantageous manner, it is thus possible to present a not yet made virtual hearing device or a corresponding other hearing device directly to the "user". In particular, for example, the hearing impaired a new hearing aid can be presented during the sales call optically. In other words, with the method according to the invention or the graphic device according to the invention, it is possible to show the user a picture or a mirror image of him including a new hearing aid or to give it to the hand, because the acoustician will certainly the device specifically suitable not available as a demonstration device.

Vorzugsweise wird der virtuelle Kopf aus digitalen Daten eines Standardkopfs und mindestens einer digitalen Fotografie des Kopfs des Nutzers oder des Teils davon gewonnen. Insbesondere kann dabei der Standardkopf aus mehreren Standardköpfen ausgewählt werden. Damit ist der Rechenaufwand zum Erstellen einer dreidimensionalen virtuellen Kopfs wesentlich reduziert.Preferably, the virtual head is obtained from digital data of a standard header and at least one digital photograph of the user's head or portion thereof. In particular, the standard head can be selected from several standard heads. Thus, the computational effort to create a three-dimensional virtual head is significantly reduced.

Vorteilhafterweise wird nun der virtuelle Kopf mit Hilfe einer Skelettanimation aus dem Standardkopf erzeugt. Diese Skelettanimation trägt nochmals wesentlich zur Reduktion des Rechenaufwands bei.Advantageously, the virtual head is now generated by means of a skeleton animation from the standard header. This skeleton animation contributes significantly to the reduction of the computational effort.

Darüber hinaus ist es für das Gestalten des virtuellen Kopfs günstig, mit Hilfe einer Bilderkennung die Form und/oder die Lage einer vorgegebenen Stelle der Fotografie zu ermitteln. So können beispielsweise das Kinn und die Wangenknochen automatisch erkannt und der Standardkopf entsprechend der Fotografie automatisch verändert werden.Moreover, it is favorable for the design of the virtual head to determine with the aid of an image recognition the shape and / or the position of a given location of the photograph. For example, the chin and cheekbones can be detected automatically and the standard head can be automatically changed according to the photograph.

Die virtuelle Hörvorrichtung kann beispielsweise aus einer Datenbank mit mehreren derartigen Hörvorrichtungen bereitgestellt werden. Damit kann ohne Rechenaufwand eine Vorauswahl oder eine endgültige Wahl der Hörvorrichtung getroffen werden.The virtual hearing device can be provided, for example, from a database with a plurality of such hearing devices. This can be done without any computational effort, a preselection or a final choice of the hearing.

Bei der Anpassung beispielsweise von IdO-Hörgeräten ist es von Vorteil, wenn die entsprechende virtuelle Hörvorrichtung durch Scannen des Gehörgangs des Nutzers mit anschließender 3D-Modellbildung gewonnen wird. Damit kann insbesondere ein Eindruck gewonnen werden, wie weit das Hörgerät in den Gehörgang gesteckt werden kann.When adapting, for example, ITE hearing aids, it is advantageous if the corresponding virtual hearing device is obtained by scanning the auditory canal of the user with subsequent 3D model formation. In particular, an impression can be gained as to how far the hearing device can be inserted into the auditory canal.

Die vorliegende Erfindung wird anhand der beigefügten Zeichnungen näher erläutert, in denen zeigen:

FIG 1
den prinzipiellen Aufbau eines Hörgeräts gemäß dem Stand der Technik;
FIG 2
eine Skizze der Situation bei einem Akustiker mit 3D-Darstellung eines Kopfs;
FIG 3
einen vergrößerten Ausschnitt aus FIG 2;
FIG 4
eine Skizze zur fotografischen Aufnahme eines Ohrbereichs eines Nutzers;
FIG 5
eine Bildschirmansicht eines Ohrbereichs mit virtuellem Hörgerät von der Seite;
FIG 6
die dreidimensionale Ohransicht mit virtuellem Hörgerät von vorne und
FIG 7
die dreidimensionale Ohransicht mit virtuellem Hörgerät von hinten.
The present invention will be further explained with reference to the accompanying drawings, in which:
FIG. 1
the basic structure of a hearing aid according to the prior art;
FIG. 2
a sketch of the situation in an acoustician with 3D representation of a head;
FIG. 3
an enlarged section FIG. 2 ;
FIG. 4
a sketch for photographing an ear portion of a user;
FIG. 5
a screen view of an ear area with virtual hearing aid from the side;
FIG. 6
the three-dimensional ear view with virtual hearing aid from the front and
FIG. 7
the three-dimensional ear view with virtual hearing aid from behind.

Die nachfolgend näher geschilderten Ausführungsbeispiele stellen bevorzugte Ausführungsformen der vorliegenden Erfindung dar.The embodiments described in more detail below represent preferred embodiments of the present invention.

In FIG 2 ist eine Situation dargestellt, in der ein Hörgeschädigter 10 einen Akustiker 11 aufsucht, um sich ein Hörgerät anpassen zu lassen. Um dem Hörgeschädigten eine möglichst reale Vorstellung des neuen bzw. zukünftigen Hörgeräts zu verschaffen, erstellt der Akustiker 11 vom Kopf des Hörgeschädigten 10 ein virtuelles 3D-Abbild. Hierzu benötigt der Akustiker für mindestens drei 2D-Aufnahmen bzw. -Fotografien des Kopfes. Für die Aufnahmen genügen handelsübliche Webcams oder auch Digicams. Zur Schaffung eines 3D-Modells des Kopfs bedient man sich günstigerweise einer Datenbank mit Standardköpfen, aus denen dann ein naturgetreues Abbild des Kopfs des Hörgeschädigten 10 erzeugt werden kann. Es ist dann nur noch notwendig, einen möglichst passenden Standardkopf auszusuchen und die markanten Partien des Kopfs entsprechend den Fotografien abzuändern. Hierzu kann beispielsweise eine Bilderkennung dienen, die auf den Fotos die Ohren, die Augen, die Augenbrauen, die Lippen, die Nase oder die Wangenknochen erkennt. Anhand dieser Erkennung können dann die entsprechenden Partien des Standardkopfs in Form und Lage abgeändert werden. Hierzu ist es wiederum günstig, die bekannte Technologie der Skelettanimation (Skeletal Animation) zu verwenden, denn Veränderungen lassen sich mit weniger Rechenaufwand anhand einfacher Skelett- oder Gerüstdarstellungen durchführen. Darauf basierend lässt sich dann das endgültige 3D-Modell des Kopfes synthetisieren, was mit ebenfalls bekannten Technologien (Morphing) möglich ist.In FIG. 2 a situation is illustrated where a hearing impaired 10 visits an acoustician 11 to have a hearing aid fitted. In order to provide the hearing impaired as real as possible a presentation of the new or future hearing aid, the acoustician 11 creates a virtual 3D image from the head of the hearing impaired 10. For this, the acoustician needs at least three 2D shots or photographs of the head. For the recordings commercial webcams or even digicams are sufficient. To create a 3D model of the head is conveniently used a database of standard heads, from which then a lifelike image of the head of the hearing impaired 10 can be generated. It is then only necessary to select the most suitable standard head and to modify the striking parts of the head according to the photographs. For this purpose, for example, serve an image recognition, which recognizes in the photos the ears, the eyes, the eyebrows, the lips, the nose or the cheekbones. Based on this recognition, the corresponding sections of the standard head can then be changed in shape and position. For this purpose, it is again favorable to use the known skeletal animation technology, since changes can be carried out with less computational effort on the basis of simple skeleton or skeleton representations. Based on this, the final 3D model of the head can then be synthesized, which is also possible with well-known technologies (morphing).

Ein 3D-Modell des einzusetzenden Hörgeräts steht beispielsweise für HdO-Hörgeräte in Form von CAD-Daten in einer Datenbank zur Verfügung. Im Falle von IdO-Hörgeräten können auch Scan-Daten oder die Daten von vorgefertigten IdO-Schalen als Grundlage für das jeweilige 3D-Modell dienen.A 3D model of the hearing device to be used is available, for example, for BTE hearing aids in the form of CAD data in a database. In the case of IdO hearing aids, scan data or the data of prefabricated IdO shells can serve as the basis for the respective 3D model.

Nun stehen zwei 3D-Modelle, das des Kopfes und das des Hörgeräts, zur Verfügung, die es ineinander zu fügen gilt, um den virtuellen Raum zu erstellen. Für diesen Schritt wird ein so genannter "Alignmentalgorithmus" eingesetzt, mit dem das Hörgerät, der Schlauch, der RIC bzw. der Hörer exakt hinter oder in das Ohr gepasst wird. Dabei werden die beiden 3D-Modelle ineinandergefügt, so dass letztlich ein 3D-Modell des virtuellen Kopfes des Hörgeschädigten 10 zusammen mit einem gewählten Hörgerät zur Verfügung steht. Aus diesem 3D-Modell lässt sich schließlich auf einem Bildschirm 12 eines Anpassgeräts, auf dem eine Anpasssoftware läuft, ein Abbild 13 gemäß FIG 2 erzeugen.Now there are two 3D models, that of the head and the hearing aid, which you have to put together to create the virtual space. For this step, a so-called "alignment algorithm" is used with which the hearing aid, the tube, the RIC or the handset is fitted exactly behind or in the ear. In this case, the two 3D models are joined together, so that ultimately a 3D model of the virtual head of the hearing impaired 10 is available together with a selected hearing aid. From this 3D model can finally be on a screen 12 of an adapter on which runs a fitting software, an image 13 according to FIG. 2 produce.

In FIG 3 ist ein Ausschnitt des Abbilds 13 vergrößert dargestellt. Es zeigt also nicht nur den virtuellen Kopf 14', sondern auch das virtuelle Hörgerät 15'. Beide, sowohl der Kopf 14' als auch das Hörgerät 15' liegen als dreidimensionaler Datensatz vor, so dass das Abbild 13 entsprechend gedreht werden kann. Damit kann der Hörgeschädigte 10 seinen Kopf 14' mit dem angebrachten Hörgerät 15' virtuell aus unterschiedlichen Perspektiven betrachten. Dabei ist es einfach möglich, den Typ, die Farbe und die Größe des virtuellen Hörgeräts 15 zu verändern und zu präsentieren.In FIG. 3 a section of the image 13 is shown enlarged. It thus shows not only the virtual head 14 ', but also the virtual hearing aid 15'. Both, both the head 14 'and the hearing aid 15' are present as a three-dimensional data set, so that the image 13 can be rotated accordingly. Thus, the hearing impaired 10 can view his head 14 'with the attached hearing aid 15' virtually from different perspectives. It is easily possible to change and present the type, the color and the size of the virtual hearing device 15.

Die FIG 4 bis 7 zeigen eine etwas vereinfachte Ausführungsform der vorliegenden Erfindung. Gemäß diesem Ausführungsbeispiel wird lediglich ein Ohrabschnitt 16 des Hörgeschädigten 10 zur Bildung des 3D-Modells eines Teils des Kopfes fotografiert bzw. aufgenommen. Im Rahmen des vorliegenden Dokuments wird auch dieser Kopfabschnitt als "virtueller Kopf" bezeichnet. Um die Aufnahme zu gewinnen, wird beispielsweise eine Webcam oder Videokamera 17 auf dem Bildschirm 18 eines PC installiert, welcher als Anpassgerät dient.The 4 to 7 show a somewhat simplified embodiment of the present invention. According to this embodiment, only an ear portion 16 of the hearing impaired 10 is photographed for forming the 3D model of a part of the head. As used herein, this header is also referred to as a "virtual head". In order to obtain the image, for example, a webcam or video camera 17 is installed on the screen 18 of a PC, which serves as an adapter.

FIG 5 zeigt nun ein Bild, das mit Hilfe einer Anpasssoftware (FSW: Fitting Software) auf dem Bildschirm 18 dargestellt werden kann. In einem Bildabschnitt 19 ist ein 3D-Video als virtueller Kopfabschnitt 16' mit einem virtuellen Hörgerät 20' dargestellt. Es ist dort auch zu erkennen, wie der virtuelle Hörerschlauch 21' des virtuellen HdO-Hörgeräts 20' um die virtuelle Ohrmuschel 22' geführt ist und in ein virtuelles Ohrpassstück 23' im Zentrum der virtuellen Ohrmuschel 22' mündet. FIG. 5 now shows an image that can be displayed on the screen 18 using a fitting software (FSW: Fitting Software). In an image section 19 is a 3D video as a virtual head section 16 'with a virtual hearing aid 20 'shown. It can also be seen there how the virtual receiver tube 21 'of the virtual BTE hearing device 20' is guided around the virtual auricle 22 'and opens into a virtual earmold 23' in the center of the virtual auricle 22 '.

In FIG 5 ist der virtuelle Kopfabschnitt 16' von der Seite dargestellt. Es kann aber auch gemäß FIG 6 ein Anblick des virtuellen Ohrabschnitts 16' von vorne präsentiert werden. Ebenso lässt sich gemäß FIG 7 darstellen, wie das Hörgerät 20' im getragenen Zustand von hinten aussieht.In FIG. 5 the virtual head section 16 'is shown from the side. But it can also according to FIG. 6 a sight of the virtual ear portion 16 'are presented from the front. Likewise, according to FIG. 7 represent how the hearing aid 20 'looks in the worn state from behind.

Die virtuellen Darstellungen können entweder als Einzelaufnahmen wie in den FIG 5 bis 7 oder im Rahmen eines bewegten Videofilms präsentiert werden. Dabei ist es - wie erwähnt - möglich, das Modell oder die Farbe des dargestellten Hörgeräts zu verändern. Insbesondere können so also auch HdO-, IdO- oder CIC-Hörgeräte virtuell präsentiert werden. Anhand dieser dreidimensionalen Darstellungen kann dann der Hörgeschädigte besser entscheiden, für welchen Hörgerätetyp bzw. welche Farbe er sich entscheidet. Damit steigt die Zufriedenheit beim Tragen des Hörgeräts zumindest im Hinblick auf das Erscheinungsbild.The virtual representations can either be as single shots as in the FIGS. 5 to 7 or presented as part of a moving video film. It is - as mentioned - possible to change the model or the color of the hearing aid shown. In particular, thus also BTE, ITE or CIC hearing aids can be presented virtually. Based on these three-dimensional representations, the hearing-impaired person can then better decide which type of hearing aid or which color he chooses. This increases the satisfaction when wearing the hearing aid, at least in terms of appearance.

Claims (10)

Verfahren zur Darstellung einer Hörvorrichtung an einem Kopf eines Nutzers (10) gekennzeichnet durch - Bereitstellen eines 3D-Modells der Hörvorrichtung als virtuelle Hörvorrichtung (15, 20'), - Gewinnen eines 3D-Modells des Kopfs des Nutzers (10) oder eines Teils davon als virtuellen Kopf (14', 16'), - Ausrichten der virtuellen Hörvorrichtung (15', 20') an dem virtuellen Kopf (16) und - grafisches Darstellen des virtuellen Kopfs (14', 16') mit der eingepassten virtuellen Hörvorrichtung (15', 20'). A method for displaying a hearing device on a head of a user (10) characterized by Providing a 3D model of the hearing device as a virtual hearing device (15, 20 '), Obtaining a 3D model of the head of the user (10) or a part thereof as a virtual head (14 ', 16'), Aligning the virtual hearing device (15 ', 20') on the virtual head (16) and - Graphically representing the virtual head (14 ', 16') with the fitted virtual hearing device (15 ', 20'). Verfahren nach Anspruch 1, wobei der virtuelle Kopf (14', 16') aus digitalen Daten eines Standardkopfs und mindestens einer digitalen Fotografie des Kopfs des Nutzers (10) oder eines Teils davon gewonnen wird.The method of claim 1, wherein the virtual head (14 ', 16') is obtained from digital data of a standard header and at least one digital photograph of the user's head (10) or a part thereof. Verfahren nach Anspruch 2, wobei der Standardkopf aus mehreren Standardköpfen ausgewählt wird.The method of claim 2, wherein the standard head is selected from a plurality of standard heads. Verfahren nach Anspruch 2 oder 3, wobei der virtuelle Kopf (14', 16') mit Hilfe einer Skelettanimation aus dem Standardkopf erzeugt wird.Method according to claim 2 or 3, wherein the virtual head (14 ', 16') is generated by means of a skeleton animation from the standard head. Verfahren nach einem der Ansprüche 2 bis 4, wobei mit Hilfe einer Bilderkennung die Form und/oder Lage einer vorgegebenen Stelle der Fotografie ermittelt und für ein Gestalten des virtuellen Kopfs (14', 16') herangezogen wird.Method according to one of claims 2 to 4, wherein the shape and / or position of a predetermined position of the photograph determined by means of an image recognition and used for a design of the virtual head (14 ', 16'). Verfahren nach einem der vorhergehenden Ansprüche, wobei die virtuelle Hörvorrichtung aus einer Datenbank mit mehreren derartigen Hörvorrichtungen bereitgestellt wird.Method according to one of the preceding claims, wherein the virtual hearing device is provided from a database with a plurality of such hearing devices. Verfahren nach einem der vorhergehenden Ansprüche, wobei die virtuelle Hörvorrichtung durch Scannen des Gehörgangs des Nutzers (10) mit anschließender 3D-Modellbildung gewonnen wird.Method according to one of the preceding claims, wherein the virtual hearing device is obtained by scanning the auditory canal of the user (10) with subsequent 3D modeling. Grafikvorrichtung zur Darstellung einer Hörvorrichtung an einem Kopf eines Nutzers (10) gekennzeichnet durch - eine Speichereinrichtung zum Bereitstellen eines 3D-Modells der Hörvorrichtung als virtuelle Hörvorrichtung (15', 20'), - eine Optikeinrichtung (17) zum Gewinnen eines 3D-Modells des Kopfs des Nutzers (10) oder eines Teils davon als virtuellen Kopf (14', 16'), - eine Recheneinrichtung zum Ausrichten der virtuellen Hörvorrichtung (15', 20') an dem virtuellen Kopf (14', 16') und - eine Grafikeinrichtung (18) zum grafischen Darstellen des virtuellen Kopfs (14', 16') mit der eingepassten virtuellen Hörvorrichtung (15', 20'). Graphic device for displaying a hearing device on a head of a user (10) characterized by a memory device for providing a 3D model of the hearing device as a virtual hearing device (15 ', 20'), an optical device (17) for obtaining a 3D model of the head of the user (10) or a part thereof as a virtual head (14 ', 16'), a computing device for aligning the virtual hearing device (15 ', 20') on the virtual head (14 ', 16') and - A graphics device (18) for graphically displaying the virtual head (14 ', 16') with the fitted virtual hearing device (15 ', 20'). Grafikvorrichtung nach Anspruch 8, wobei die Optikeinrichtung (17) zum Erstellen einer digitalen Fotografie des Kopfs des Nutzers (10) oder des Teils davon ausgelegt ist, so dass der virtuelle Kopf (14', 16') aus digitalen Daten eines Standardkopfs und der digitalen Fotografie erstellbar ist.The graphics device of claim 8, wherein the optical device (17) is adapted to take a digital photograph of the user's head (10) or part thereof, such that the virtual head (14 ', 16') consists of standard head digital data and the digital head Photography is buildable. Grafikvorrichtung nach Anspruch 8 oder 9, wobei die Optikeinrichtung (17) einen Scanner aufweist, mit dem dreidimensionale Daten des Gehörgangs des Nutzers (10) für die Bildung der virtuellen Hörvorrichtung (15', 20') bereitstellbar sind.A graphic device according to claim 8 or 9, wherein the optical device (17) comprises a scanner with which three-dimensional data of the auditory canal of the user (10) for the formation of the virtual hearing device (15 ', 20') can be provided.
EP08164382A 2007-11-12 2008-09-16 Method for three-dimensional representation of a hearing device on a head and corresponding graphic device Active EP2059068B1 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
DE102007053832A DE102007053832A1 (en) 2007-11-12 2007-11-12 Method for three-dimensional representation of a hearing device on a head and corresponding graphics device

Publications (2)

Publication Number Publication Date
EP2059068A1 true EP2059068A1 (en) 2009-05-13
EP2059068B1 EP2059068B1 (en) 2012-08-08

Family

ID=40344393

Family Applications (1)

Application Number Title Priority Date Filing Date
EP08164382A Active EP2059068B1 (en) 2007-11-12 2008-09-16 Method for three-dimensional representation of a hearing device on a head and corresponding graphic device

Country Status (4)

Country Link
US (1) US8249261B2 (en)
EP (1) EP2059068B1 (en)
DE (1) DE102007053832A1 (en)
DK (1) DK2059068T3 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CH711767A2 (en) 2017-02-23 2017-05-15 Sonova Ag Method for soundproofing of hard-of-hearing customers at a sales, consulting and / or administrative office and system for carrying out the method.
US11166115B2 (en) 2018-10-18 2021-11-02 Gn Hearing A/S Device and method for hearing device customization

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5056204A (en) 1989-05-17 1991-10-15 Ascom Audiosys Ag Method of producing hearing aids
WO1999023609A1 (en) 1997-10-30 1999-05-14 Headscanning Patent B.V. A method and a device for displaying at least part of the human body with a modified appearance thereof
EP1246506A1 (en) * 2001-03-26 2002-10-02 Widex A/S A CAD/CAM system for designing a hearing aid
US20030123026A1 (en) 2000-05-18 2003-07-03 Marc Abitbol Spectacles fitting system and fitting methods useful therein
US6700586B1 (en) * 2000-08-23 2004-03-02 Nintendo Co., Ltd. Low cost graphics with stitching processing hardware support for skeletal animation
US20040165740A1 (en) 2002-12-19 2004-08-26 Tong Fang Interactive binaural shell modeling for hearing aids
US20050088435A1 (en) 2003-10-23 2005-04-28 Z. Jason Geng Novel 3D ear camera for making custom-fit hearing devices for hearing aids instruments and cell phones
JP2007108254A (en) * 2005-10-11 2007-04-26 Towa Kogyo:Kk Eyeglasses with hearing aid and adjusting method thereof

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DK2335426T3 (en) * 2008-10-10 2017-08-28 Widex As PROCEDURE FOR THE MANUFACTURING OF A HEARING WHICH HAS AN INDIVIDUALLY CUSTOMIZED ELASTIC COMPONENT

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5056204A (en) 1989-05-17 1991-10-15 Ascom Audiosys Ag Method of producing hearing aids
WO1999023609A1 (en) 1997-10-30 1999-05-14 Headscanning Patent B.V. A method and a device for displaying at least part of the human body with a modified appearance thereof
US20030123026A1 (en) 2000-05-18 2003-07-03 Marc Abitbol Spectacles fitting system and fitting methods useful therein
US6700586B1 (en) * 2000-08-23 2004-03-02 Nintendo Co., Ltd. Low cost graphics with stitching processing hardware support for skeletal animation
EP1246506A1 (en) * 2001-03-26 2002-10-02 Widex A/S A CAD/CAM system for designing a hearing aid
US20040165740A1 (en) 2002-12-19 2004-08-26 Tong Fang Interactive binaural shell modeling for hearing aids
US20050088435A1 (en) 2003-10-23 2005-04-28 Z. Jason Geng Novel 3D ear camera for making custom-fit hearing devices for hearing aids instruments and cell phones
JP2007108254A (en) * 2005-10-11 2007-04-26 Towa Kogyo:Kk Eyeglasses with hearing aid and adjusting method thereof

Also Published As

Publication number Publication date
DK2059068T3 (en) 2012-12-10
DE102007053832A1 (en) 2009-05-14
US8249261B2 (en) 2012-08-21
US20090123001A1 (en) 2009-05-14
EP2059068B1 (en) 2012-08-08

Similar Documents

Publication Publication Date Title
DE102008017194B4 (en) Hearing device for wearing in the auricle with an individual molded part and method for producing a hearing device
US10021496B2 (en) Method and system for creating non-occluding earpieces
DE102014216086A1 (en) Device and method for integrating hearing device components
EP2023668B1 (en) Hearing aid with visualised psycho-acoustic magnitudes and corresponding method
EP3528513B1 (en) Method for adjusting the parameters of a hearing system
DE102005046169A1 (en) Hearing aid with an antenna
EP2172062B1 (en) Method for adapting a hearing device using a perceptive model
EP2919486A1 (en) Method for producing hearing aid fittings
EP2620940A1 (en) Method and hearing device for estimating a component of one's own voice
EP2432253A1 (en) Connector for hearing aid, hearing aid and hearing aid system
US8319151B2 (en) Method to manufacture a hearing aid with indirect laser exposure
EP2059068B1 (en) Method for three-dimensional representation of a hearing device on a head and corresponding graphic device
EP2227039A2 (en) Hearing aid with an acoustical unit and a muffler section
DE102015212609A1 (en) Method for operating a hearing aid system and hearing aid system
DE102011078675B4 (en) Impression structure for a hearing device
DE102007033896B4 (en) Hearing apparatus with signal processing on the basis of design-related parameters and corresponding method
EP4125278A1 (en) Method for determining the necessary or ideal length of a cable in a hearing aid
DE102018120085B4 (en) Procedure for fine-tuning an earphone and earphone
US10325404B1 (en) 3D virtual automated modeling of custom concha module
WO2013020851A1 (en) Production of an earpiece by thermoforming
DE102016224282A1 (en) Housing for a hearing aid
DE102016212881A1 (en) Method for the interactive determination of setting parameters of a hearing aid
DE102012207316B3 (en) Method of making an earpiece with vent
DE102007012075B4 (en) Method for designing and making a shell of a hearing device
Harder Individualized directional microphone optimization in hearing aids based on reconstructing the 3D geometry of the head and ear from 2D images

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA MK RS

17P Request for examination filed

Effective date: 20090720

17Q First examination report despatched

Effective date: 20090814

AKX Designation fees paid

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: AT

Ref legal event code: REF

Ref document number: 570266

Country of ref document: AT

Kind code of ref document: T

Effective date: 20120815

Ref country code: CH

Ref legal event code: NV

Representative=s name: SIEMENS SCHWEIZ AG

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502008007881

Country of ref document: DE

Effective date: 20120927

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20120808

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

Effective date: 20120808

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120808

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120808

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120808

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20121208

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120808

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20121108

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120808

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120808

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120808

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20121109

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120808

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20121210

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120808

BERE Be: lapsed

Owner name: SIEMENS MEDICAL INSTRUMENTS PTE. LTD.

Effective date: 20120930

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120808

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120808

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120808

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120930

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20121119

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120808

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120808

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

26N No opposition filed

Effective date: 20130510

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120930

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20121108

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120916

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502008007881

Country of ref document: DE

Effective date: 20130510

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120808

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120808

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120916

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080916

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 570266

Country of ref document: AT

Kind code of ref document: T

Effective date: 20130916

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20130916

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 502008007881

Country of ref document: DE

Representative=s name: FDST PATENTANWAELTE FREIER DOERR STAMMLER TSCH, DE

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 502008007881

Country of ref document: DE

Representative=s name: FDST PATENTANWAELTE FREIER DOERR STAMMLER TSCH, DE

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 502008007881

Country of ref document: DE

Representative=s name: FDST PATENTANWAELTE FREIER DOERR STAMMLER TSCH, DE

Ref country code: DE

Ref legal event code: R081

Ref document number: 502008007881

Country of ref document: DE

Owner name: SIVANTOS PTE. LTD., SG

Free format text: FORMER OWNER: SIEMENS MEDICAL INSTRUMENTS PTE. LTD., SINGAPORE, SG

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 9

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 10

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 11

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: CH

Payment date: 20231001

Year of fee payment: 16

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240820

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DK

Payment date: 20240820

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20240820

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20240820

Year of fee payment: 17