WO2014081287A1

WO2014081287A1 - A camera unit adaptor, a method of recording an image and a computer program product

Info

Publication number: WO2014081287A1
Application number: PCT/NL2013/050799
Authority: WO
Inventors: Robert Michiel METSELAAR
Original assignee: Erasmus University Medical Center Rotterdam
Priority date: 2012-11-08
Filing date: 2013-11-08
Publication date: 2014-05-30
Also published as: WO2014073950A1

Abstract

The invention relates to a camera unit adaptor for coupling an add¬ on camera unit on a portable electronic device 41 to an endoscope 30 providing an image of less accessible structures. The adaptor includes a frame 13 attachable, in a pre-specified lateral position, to a front side of a camera unit. The frame is also connectable to a proximal end of an endoscope. Further, the frame may include a strip 14 of light guiding material being provided with an opening 15. The strip covers a region of the camera unit's front side where an optical sensor 3 and a flash light source 4 of the camera unit are located. The opening is aligned with the camera unit's optical sensor when the frame is attached to the front side of the camera unit, such that light from the flash light source 4 enters into the strip 14, propagates towards the strip opening 15 and is incident on the optical sensor 3 in a border zone surrounding the endoscope image of interest.

Description

AN ADAPTER FOR COUPLING A CAMERA UNIT TO AN ENDOSCOPE, A METHOD OF RECORDING AN IMAGE AND A COMPUTER PROGRAM PRODUCT

The invention relates to camera unit adaptor for coupling an addon camera unit on a portable electronic device to an endoscope providing an image of less accessible structures.

In a clinical context, dedicated photography devices are known for recording images via endoscopes. However, such devices are relatively

complex to handle and operate, heavy and expensive.

Application of a digital compact camera is also cumbersome since most compact camera's are provided with a zoom lens requiring a complex, adjustable connection to the endoscope.

When taking a picture using an add-on camera that is provided on a portable electronic device, such as a mobile phone, a smart phone or a tablet, a desired image from the endoscope can be over exposed.

It is an object of the invention to provide camera unit adaptor, wherein the above-mentioned disadvantage is reduced. In particular, the invention aims at obtaining a camera adaptor that enables convenient

photography of endoscope images using a camera functionality on

commercially available portable electronic devices. Thereto, according to an aspect of the invention, a camera unit adaptor is provided according to claim 1.

By applying the flash light source in combination with the strip of light guiding material, pixels of the recorded image surrounding the

endoscope image of interest are illuminated, so that the light intensity

range of the pixels associated with an image portion of the endoscope on the one hand, and the pixels associated with an image portion surrounding the endoscope image reduces. Then, an effect of over exposure is also reduced since the camera unit may store the image pixels more efficient, assigning available bits to pixel values in said reduced intensity range. Further, by providing a frame for attachment to a camera unit and for connection to a proximal end of an endoscope, proper alignment of the optical sensor, the strip, the strip opening and the endoscope can easily be realized, requiring less skills to hold both the endoscope and a camera unit.

In principle, the camera unit adaptor can be used with any add-on camera unit on a portable electronic device, so that medical personal can easily make pictures with commercially available cameras on mobile phones, smart phones etc. Such images can be recorded in a clinical setting. In addition, also other professionals using an endoscope may apply a camera unit adaptor according to the invention for making images.

The invention also relates to a method of recording an image.

Further, the invention relates to a computer program product. A computer program product may comprise a set of computer executable instructions stored on a data carrier, such as a flash memory, a CD or a DVD. The set of computer executable instructions, which allow a

programmable computer to carry out the method as defined above, may also be available for downloading from a remote server, for example via the Internet, e.g. as an app.

Other advantageous embodiments according to the invention are described in the following claims.

By way of example only, embodiments of the present invention will now be described with reference to the accompanying figures in which

Fig. 1 shows a schematic perspective view of a front side of a camera module;

Fig. 2 shows a schematic perspective view of a front side of a camera module adaptor according to the invention;

Fig. 3a shows a schematic cross sectional view of the camera module adaptor shown in Fig. 2 attached to a camera module; Fig. 3b shows a strip of light guiding material in the camera module adaptor of Fig. 3a;

Fig. 3c shows a schematic cross sectional view of another camera module adaptor attached to a camera module;

Fig. 3d shows an illuminating light path structure of the adaptor shown in Fig. 3c in more detail;

Fig. 4 shows an image recorded by the camera module as shown in Fig. 1;

Fig. 5 shows a graph of light intensities of pixels in the image shown in Fig. 4; and

Fig. 6 shows a flow chart of an embodiment of a method according to the invention.

The figures are merely schematic views of a preferred

embodiment according to the invention. In the figures, the same reference numbers refer to equal or corresponding parts.

Figure 1 shows a schematic perspective view of a front side 2 of a camera module 1. The camera module 1 is provided with an optical sensor 3 and a flash light source 4, located within a region 5 of the camera unit's front side 2. The camera module 1 is an add-on unit on a portable electronic device 41 such as a mobile phone, a smart phone or a tablet. The front side 2 may include further structures, such as apertures of a built-in loud speaker unit or switches for operating the electronic device 41. As shown, the front side 2 of the camera module 1 has a generally rectangular design, including a top edge 6, a bottom edge 7, a first side edge 8 and a second side edge 9. However, the geometry can deviate from the embodiment shown in Fig. 1. As an example, the front side 2 may have rounded corners. Further, the surface of the shown front side 2 is mainly flat. As an alternative, the front side 2 may have, in principle, a non-flat surface, e.g. a curved surface. For ease of reference, an orthogonal coordinate system is oriented such that the front side 2 of the camera unit 1 extends in a x-direction and a y-direction wherein a z-coordinate, in the front direction of the camera unit 1, is constant.

The optical sensor 3 can be implemented as a semiconductor device such as a CMOS or CCD unit.

Figure 2 shows a schematic perspective view of a front side 12 of a camera module adaptor 11 according to the invention. The camera module adaptor 11 is provided with a frame 13 that is attachable to the front side 2 of the camera unit 1 shown in Fig. 1. The front side 12 has mainly the same geometry as the front side 2 of the camera unit 1. In the shown embodiment, the geometry is generally rectangular, and the surface of the front side 12 of the camera module adaptor 11 is generally flat, so that the front side 12 of the adaptor 11 may follow the surface of the camera module's front side 2. The front side 12 of the adaptor 11 has a top edge 16, a bottom edge 17, a first side edge 18 and a second side edge 19.

The frame 13 includes a strip 14 of light guiding material. The strip 13 is provided with an opening 15. When attached to the camera unit 1, the frame 13 of the adaptor 11 is located in a pre-specified lateral position with respect to the front side 2 of the camera unit 1, i.e. the adaptor 11 is fixed in a direction x and a direction y with respect to the front side 2 of the camera unit 1. Then, the strip 14 covers the region 5 of the camera unit 1 where the light sensor 3 and the flash light source 4 are located. Further, the opening 15 is then aligned with the optical sensor 3 of the camera unit 1.

The frame 13 further includes a connector element 20 for connection to a proximal end 30 of an endoscope.

Figure 3a shows a schematic cross sectional view of the camera module adaptor 11 attached to the camera module 1. The cross sectional view is taken along a cross section along a transverse plane A extending in the x-direction and z-direction, through the strip 14 and transverse to the orientation of the front side 12 of the adaptor 11, see Fig. 2. The connector element 20 includes an opening 21 aligned with the opening 15 of the strip 14. The connector element 20 includes an annular element for clampingly receiving a proximal end 30 of the endoscope, thus providing a snap connection. The frame 13 is further provided with clamping elements 23 clampingly surrounding the first and second side edges 8, 9 of the front side 2 of the camera unit 1.

It is noted that the frame 13 can be attached to the camera unit 1 in various other ways, e.g. by clamping the top edge 6 and the bottom edge 7 of the camera unit's front side 2, and/or by using another mechanism for attaching the frame to the camera unit such as a magnetic connection, a screw connection and/or by surrounding not merely the side edges 7, 8 of the camera unit, but also a back side of the camera unit e.g. using an envelope structure.

When the frame 13 is assembled to the camera unit 1 and to the proximal end 30 of the endoscope, the proximal end 30 of the endoscope is aligned with the strip opening 15 of the strip and with the optical sensor 3 of the camera unit 1.

As also shown in Fig. 3a, the flash light source 4 is integrated in the front side 2 of the camera unit 1. The flash light source 4 can be implemented as a LED or a multiple number of LED's. The light of the flash light source 4 propagates through a lens element 24 when leaving the camera unit 1. External light L directed to the optical sensor 3 travels through a window 25 and a lens module 26 before reaching the optical sensor 3. The lens module 26 generally includes a lens having a fixed focal length. The flash light source 4, the lens element 24, the optical sensor 3, the lens module 26 and the window 25 are included in the camera unit 1 that is integrated in the portable electronic device 41.

Fig. 3b shows the strip 14 of light guiding material in the camera module adaptor 11 in more detail. In Fig. 3b, the top part shows the strip 14 in a front view, while the bottom part shows the strip 14 in a cross sectional view, at the level of the opening 15. Preferably, the light guiding material in the strip 14 includes a transparent thermoplastic such as PMMA. However, also other light guiding material could be applied, e.g. glass. In the shown embodiment, the strip 14 is at least partially coated, on the side facing away from the camera unit 1, with a light impenetrable coating layer 27 to counteract that light propagating through the strip 14 leaks away, in a direction away from the camera unit 1. The operation of the flash light source is controlled by a control unit for turning on the flash light source. The control unit can be implemented as processor loaded with specific software driving the flash light source.

In the embodiment shown in Fig. 3a,b the strip 14 of light guiding material forms an illuminating light path structure defining, in the attached state of the camera unit adaptor 11, an illuminating hght path extending from the flash light source 4 of the camera module 1 towards the optical sensor 3 of the camera module 1. Similarly, the opening 15 in the strip 14 of light guiding material forms an image light path structure defining, in the attached state, an image light path section for light traveling from the proximal end 30 of the endoscope towards the optical sensor 3 of the camera module 1. In the shown embodiment, the image hght path section is part of an image hght path extending between the proximal end 30 of the endoscope and the optical sensor 3 of the camera module 1, the image light path including the opening 15 in the strip 14 and the opening 21 of the connector element 20.

Figure 3c shows a schematic cross sectional view of another camera module adaptor 11 attached to the camera module 1. Figure 3d shows the illuminating light path structure of the frame 13 of the adaptor 11 in more detail. In Fig. 3d, the top part shows the strip 14 in a front view, while the bottom part shows the strip 14 in a cross sectional view, at the level of the opening 15. Here, the illuminating light path structure is not implemented as a strip 14 of light guiding material, but includes a reflecting surface 27' that is located offset and opposite to the front side of the camera module 1, in the attached state of the adaptor 11. In the shown embodiment, the illuminating light path structure includes a sheet or plate 80 provided with an opening 15 forming the image light path structure. The sheet or plate 80 is corrugated with respect to a backside B of the frame 13 facing the front side 2 of the portable electronic device 41. On the side facing the front side 2 of the portable electronic device 41, the sheet or plate 80 is provided with the above-mentioned reflecting surface 27'. On the opposite side, facing away from the front side 2 of the portable electronic device 41, the sheet or plate 80 is provided with a light impenetrable coating layer 27 to counteract that light propagating from the flashlight 4 leaks away, in a direction away from the camera unit 1. The reflecting surface 27' is e.g. formed by a coating having a high reflection coefficient, e.g. in a light colour. Further, by arranging the reflecting surface 27' such that it is offset from the front side 2 of the portable electronic device 41, light may travel from the flashlight 4 towards the sensor 3 of the camera unit 1, e.g. via single or multiple reflection against the reflecting surface 27' and the outside surface of the portable electronic device 41. It is noted that also other variants of the illuminating light path structure are possible, e.g. using an optical system including lenses or other optical components.

As shown in Fig. 3c, an external light source generates light L' entering the endoscope via a light port LP and propagating via an endoscope channel CH towards a structure S to be examined. Then, an endoscope image L is formed travelling via the endoscope channel CH and the image light path section towards the sensor 3 of the camera unit 1. The endoscope image L is mapped on the sensor 3 that is provided with sensor pixels recording the endoscope image L. Generally, the sensor includes first pixels on which the endoscope image L is mapped, and second pixels that are located outside the mapped endoscope image. In the embodiments described above, the illuminating light path is arranged for illuminating sensor elements in the sensor that are shielded from light propagating via the image light path section. Then, the second pixels of the sensor, e.g. located in a border region of the image recorded by the sensor, are illuminated by the flashlight 4. By illuminating sensor elements that surround first sensor elements recording the image light, i.e. light propagating from the

endoscope, the dynamic range of the sensor is used more efficiently as described in more detail below.

Figure 4 shows an image recorded by the camera module 1. Since the endoscope has a substantially circular cross sectional profile, the image 40 contains a relatively small disc shaped image zone 41 associated with visual information from the endoscope interior, and a relatively large border image zone 42 surrounding the disc shaped image part zone 41. The border image zone 42 represents optic information of structures outside the endoscope interior, e.g. wall portions of the endoscope and/or frame parts of the adaptor 11. Usually, such optic information is not of interest for the user and does not contain visible information of interest. When applying a lens having a fixed focal length, no optical zoom function is available, so that the relative size of the image part zone 41 with respect to the border image zone 42 is fixed.

When the endoscope is provided with a light source for illuminating the structures at a distal end of the endoscope, the light intensities of the pixels in the disc shaped image zone 41 are generally high, while light intensities of the pixels in the border zone 42 are generally much lower.

Figure 5 shows a graph of light intensities L of pixels in the image shown in Fig. 4. The graph visualizes in a first graph section 43 the light value of structures represented by image pixels Pi in the disc shaped image part 42. Similarly, the graph visualizes in a second graph section 44, 45 the light value of structures represented by border pixels Pb in the border image zone 42. Due to the illumination of the endoscope, the image pixels Pi have a relatively high value, while the border pixels Pb generally have a relatively low value, see the lower second graph section 44. The intensity levels of the image pixels Pi and the border pixels Pb are present in a first range Ri.

Since the dynamic range of the optical sensor 3 is limited, the image pixel Pi values are than represented by extreme high values while the border pixels Pb are represented by extreme low values. In other words, the disc shaped image zone 41 is nearly white while the border image part 42 is nearly black. The image zone is over exposed.

According to an aspect of the invention, the flash light source 4 is switched on when the image from the endoscope is recorded by the camera unit 1. Then, the light F from the flash light source 4 enters into the strip 14, propagates towards the strip opening 15 and is incident on the optical sensor 3, in the border zone 42, thereby contributing to a higher light intensity level of the border pixels Pb. The intensity of the border pixels Pb raises to higher levels as represented by the higher second graph section 45.

Since the intensity levels of the image pixels Pi and the border pixels Pb are now with a second range R2, significant smaller than the first range Ri, the dynamic range of the optical sensor is used more efficiently. Advantageously, more bits are now available for the representation of the image pixels Pi, providing a meaningful image of the image pixels Pi in the image zone 41.

Advantageously, the light intensity of the flash light source 4 is set such that light F entering the optical sensor 3 via the strip 14 and the light L entering the optical sensor 3 via the proximal end 30 of the endoscope are in the same order of magnitude.

Further, the colour of the light F entering the optical sensor 3 via the strip 14 is optionally set such that said colour is similar to the colour of the light L entering the optical sensor 3 via the proximal end 30 of the endoscope. As a further option, not merely a single image is recorded, but a multiple number of images. In principle, the multiple images can be taken with short intermediate time intervals, obtaining a sequence of images forming a video.

It is noted that also a camera unit adaptor provided with another implementation of the illuminating light path, e.g. including a reflecting surface that is located offset and opposite to the front side of the camera unit, as described above, contributes to a higher light intensity level of the border pixels Pb, thereby using the dynamic range of the optical sensor more efficiently. By using the above-described camera unit adaptor in

combination with an endoscope, images can be recorded from structures that are less accessible, e.g. in the medical field or in the non-medical field of remote analysis of structures. The endoscope can be either rigid or flexible. In combination with a camera unit on a mobile (smart) phone, quick and convenient photography is available.

As an example of clinical, medical applications, it is noted that the adaptor can be used for diagnosis / surgery via ENT related apertures in the human body, e.g. for visualizing the ear drum or paranasal sinus, or for inspection of stomach or other intestine organs. The images can be taken either in an outpatient setting or on any location in or outside a hospital. As an example of non-medical applications, it is noted that the adaptor can be used for inspection of engine parts that are visually less accessible.

Figure 6 shows a flow chart of an embodiment of the method according to the invention. The method is used for recording an image. The method comprises a step of providing 110 an endoscope providing an image of less accessible structures, a step of providing 120 a camera unit including an optical sensor and a flash light source on its front side, a step of providing 130 a strip of light guiding material being provided with an opening, a step of covering 140, with the strip, a region of the camera unit's front side where the optical sensor and the flash light source of the camera unit are located, a step of aligning 150 the proximal end of the endoscope with the opening of the strip and with the optical sensor of the camera unit, a step of switching 160 on the flash light source, and a step of activating 170 the optical sensor.

The method of recording an image can be performed using dedicated hardware structures, such as FPGA and/or ASIC components. Otherwise, the method can also at least partially be performed using a computer program product comprising instructions for causing a processor of a computer system or a control unit to perform the above described steps of the method according to the invention, or at least the step of switching on the flash hght source when the image is to be recorded. All (sub)steps can in principle be performed on a single processor. However, it is noted that at least one step can be performed on a separate processor. A processor can be loaded with a specific software module. Dedicated software modules can be provided, e.g. from the Internet.

The invention is not restricted to the embodiments described herein. It will be understood that many variants are possible.

As an example, the snap connection between the camera unit adaptor can be replaced by another connection type, e.g. a screw connection.

Other such variants will be apparent for the person skilled in the art and are considered to fall within the scope of the invention as defined in the following claims.

Claims

1. A camera unit adaptor for coupling an add-on camera unit on a portable electronic device to an endoscope providing an endoscope image of less accessible structures, the adaptor including a frame attachable, in a pre-specified lateral position, to a front side of a camera unit and

connectable to a proximal end of an endoscope, the frame including an image light path structure defining, in the attached state, an image light path section for image light travelhng from a proximal end of the endoscope towards an optical sensor of the camera unit for generating a sensor image, and an illuminating hght path structure defining, in the attached state, an illuminating light path from a flash hght source of the camera unit towards the optical sensor of the camera unit.

2. A camera unit adaptor according to claim 1, wherein the illuminating light path is arranged for illuminating sensor elements in the sensor that are shielded from light propagating via the image light path section.

3. A camera unit adaptor according to claim 1 or 2, wherein the illuminating light path is arranged for illuminating border sensor elements of the sensor, the border sensor elements surrounding sensor elements of the sensor on which the image hght is mapped.

4. A camera unit adaptor according to any of the preceding claims, wherein the illuminating hght path structure is implemented as a strip of light guiding material, covering, in the attached state, a region of the camera unit's front side where the optical sensor and the flash light source of the camera unit are located.

5. A camera unit adaptor according to claim 4, wherein the image light path structure is implemented as an opening provided in the strip of light guiding material, the opening being aligned with the camera unit's optical sensor when the frame is attached to the front side of the camera unit.

6. A camera unit adaptor according to claim 1, 2 or 3, wherein the illuminating light path structure is not implemented as a strip of light guiding material covering, in the attached state, a region of the camera unit's front side where the optical sensor and the flash light source of the camera unit are located.

7. A camera unit adaptor according to claim 1, 2 or 3, wherein the illuminating light path structure includes a reflecting surface that is located offset and opposite to the front side of the camera unit in the attached state.

8. A camera unit adaptor according to any of the preceding claims, wherein the frame includes clamping elements for clamping the camera unit.

9. A camera unit adaptor according to any of the preceding claims, wherein the frame is connectable to the proximal end of an endoscope via a clamping connection or a screwing connection.

10. A camera unit adaptor according to any of the preceding claims, wherein the light guiding material includes a transparent thermoplastic such as PMMA.

11. A camera unit adaptor according to any of the preceding claims, wherein the strip is at least partially coated, on the side facing away from the camera unit, with a light impenetrable layer.

12. A camera unit adaptor according to any of the preceding claims, wherein the proximal end of the endoscope is aligned with the camera unit's optical sensor when the frame is attached to the camera unit and connected to the endoscope.

13. A camera unit adaptor according to any of the preceding claims, wherein the endoscope is of a medical type.

14. A camera unit adaptor according to any of the preceding claims, wherein the camera unit is provided with a lens having a fixed focal length.

15. A camera unit, comprising an optical sensor, a flash light source and a camera unit adaptor according to any of the preceding claims 1-14 attached to the front side of the camera unit.

16. A camera unit according to claim 15, comprising a control unit for turning on the flash light source when an image is to be recorded.

17. A portable electronic device, such as a mobile phone, a smart phone or a tablet, comprising an add-on camera unit according to claim 9 or 10.

18. A method of recording an image, comprising the steps of:

- providing an endoscope providing an image of less accessible structures,

- providing an add-on camera unit on a portable electronic device, the camera unit including an optical sensor and a flash light source on its front side,

- defining an image light path section for image light travelling from a proximal end of the endoscope towards an optical sensor of the camera unit for generating a sensor image,

- providing an illuminating light path structure defining an illuminating light path from a flash light source of the camera unit towards the optical sensor of the camera unit,

- switching on the flash light source, and

- activating the optical sensor.

19. A method according to claim 18, wherein the step of defining an image light path section and the step of defining an illuminating light path is implemented by the substeps of:

- providing a strip of light guiding material being provided with an opening,

- covering, with the strip, a region of the camera unit's front side where the optical sensor and the flash light source of the camera unit are located,

- aligning the proximal end of the endoscope with the opening of the strip and with the optical sensor of the camera unit.

20. A method according to claim 19, wherein the light intensity of the flash light source is set such that light entering the optical sensor via the light guiding material and the light entering the optical sensor via the proximal end of the endoscope are in the same order of magnitude.

21. A method according to claim 19 or 20, wherein the colour of the light entering the optical sensor via the light guiding material is set such that said colour is similar to the colour of the light entering the optical sensor via the proximal end of the endoscope.

22. A method according to any of the claims 19-21, comprising the step of recording a video.

23. A computer program product for recording an image, the computer program product comprising computer readable code for causing a processor to perform the step of switching on a flash light source of an add-on camera unit on a mobile electronic device comprising an optical sensor, a flash light source and a camera unit adaptor according to any of claims 1-14 attached to the front side of the camera unit.