WO2012085765A2 - Visualization of uncertainty for 2-d lines - Google Patents

Visualization of uncertainty for 2-d lines Download PDF

Info

Publication number
WO2012085765A2
WO2012085765A2 PCT/IB2011/055661 IB2011055661W WO2012085765A2 WO 2012085765 A2 WO2012085765 A2 WO 2012085765A2 IB 2011055661 W IB2011055661 W IB 2011055661W WO 2012085765 A2 WO2012085765 A2 WO 2012085765A2
Authority
WO
WIPO (PCT)
Prior art keywords
uncertainty
contour
location
locations
image
Prior art date
Application number
PCT/IB2011/055661
Other languages
French (fr)
Other versions
WO2012085765A3 (en
Inventor
Javier Olivan Bescos
Original Assignee
Koninklijke Philips Electronics N.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics N.V. filed Critical Koninklijke Philips Electronics N.V.
Publication of WO2012085765A2 publication Critical patent/WO2012085765A2/en
Publication of WO2012085765A3 publication Critical patent/WO2012085765A3/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/002D [Two Dimensional] image generation
    • G06T11/20Drawing from basic elements, e.g. lines or circles
    • G06T11/206Drawing of charts or graphs
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/13Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20172Image enhancement details
    • G06T2207/20192Edge enhancement; Edge preservation

Definitions

  • the invention relates to visualizing uncertainties of object shapes plotted in 2-D images, especially contours computed from image data based on image segmentation.
  • the invention provides a system for visualizing the uncertainty of a contour within a contour uncertainty band in a 2-dimensioanl (2-D) image, the system comprising:
  • a transfer unit for applying an uncertainty transfer function, which assigns a renderable property to each location of a plurality of locations within the contour uncertainty band, based on the value of the uncertainty at said location;
  • a mixing unit for computing a final image for displaying the 2-D image and the renderable property assigned to each location of the plurality of locations within the contour uncertainty band.
  • the renderable property e.g., a color
  • the renderable property describes the uncertainty of the contour at each location of the plurality of locations within the contour uncertainty band.
  • the uncertainty may be, for example, the probability that the location is (or, in an alternative embodiment, is not) on the contour. Since the final image comprises, at each location of the plurality of locations within the contour uncertainty band, both the 2-D image value and the renderable property, which can be displayed concurrently or in an alternating manner, the system of the invention is capable of showing the uncertainty of the contour without hiding image details.
  • the mixing unit is arranged for determining a way of alternatingly displaying the 2-D image and the renderable property assigned by the uncertainty transfer function to each location of a plurality of locations within the contour uncertainty band.
  • the final image is obtained by alternatingly displaying the 2-D image and an uncertainty image defined by the renderable property assigned by the uncertainty transfer function to each location of a plurality of locations within the contour uncertainty band.
  • the display period for each image may be, for example, in the range of 0.1 to 1.0 second. This embodiment is particularly useful for illustrating uncertainties of temporal sequences of images.
  • the mixing unit is arranged for merging the
  • the merging may be accomplished by displaying the 2-D image in the red color channel R of the RGB color coding and an image defined by the renderable property assigned by the uncertainty transfer function to each location of a plurality of locations within the contour uncertainty band in the blue channel B of the RGB color coding, for example.
  • the renderable property comprises a color and a blending factor
  • the mixing unit is arranged for blending the color assigned by the uncertainty transfer function to each location of the plurality of locations within the contour uncertainty band and the 2-D image using the blending factor assigned by the uncertainty transfer function to each location of the plurality of locations within the contour uncertainty band.
  • Such blending has the effect that, at each location within the contour uncertainty band, the color indicates the contour uncertainty while the grayscale component shows image details, for example, an anatomical structure displayed in the image.
  • the uncertainty transfer function is a step function for visualizing a multiple contour uncertainty band using contours separating locations corresponding to a lower uncertainty from locations corresponding to a higher uncertainty.
  • Such error function allows a clear quantitative visualization of a discrete contour uncertainty.
  • the system further comprises an uncertainty unit for computing the uncertainty of the contour at each location of the plurality of locations within the contour uncertainty band.
  • system further comprises a contour unit for computing the contour.
  • the 2-D image is a cross-section image computed from a 3 -dimensional (3-D) image
  • the contour is a cross-section of a surface identified in the 3-D image by the cross-section plane
  • the uncertainty of the contour at each location of the plurality of locations within the contour uncertainty band is computed based on the uncertainty of the surface at each location of a plurality of locations in an uncertainty layer of the surface. This allows showing contour uncertainty in 2-D images derived from 3-D images.
  • a workstation comprising the system of the invention.
  • an image acquisition apparatus comprising the system of the invention system is provided.
  • a method of visualizing an uncertainty band of a contour within a 2-dimensioanl (2-D) image comprising:
  • an error transfer step for applying an uncertainty transfer function, which assigns a renderable property to each location of a plurality of locations within the contour uncertainty band, based on the value of the uncertainty at said location;
  • a mixing step for computing a final image for displaying the 2-D image and the renderable property assigned to each location of the plurality of locations within the contour uncertainty band.
  • the method further comprises an uncertainty step for computing the uncertainty of the contour at each location of the plurality of locations within the contour uncertainty band.
  • the method further comprises a contour step for computing the contour.
  • the invention provides a computer program product to be loaded by a computer arrangement, the computer program product comprising instructions for visualizing an uncertainty band of a contour within a 2-dimensioanl (2-D) image, the computer arrangement comprising a processing unit and a memory, the computer program product, after being loaded, providing said processing unit with the capability to carry out steps of the method of the invention.
  • Fig. 1 shows a block diagram of an exemplary embodiment of the system
  • Fig. 2 shows two exemplary images, each showing an exemplary contour
  • Fig. 3A shows a first exemplary uncertainty transfer function
  • Fig. 3B shows two exemplary final images resulting from applying the first uncertainty function shown in Fig. 3 A to the images of Fig. 2;
  • Fig. 4A shows a second exemplary uncertainty transfer function
  • Fig. 4B shows two exemplary final images resulting from applying the second uncertainty function shown in Fig. 4A to the images of Fig. 2;
  • FIG. 5 schematically shows an exemplary flowchart of the method
  • Fig. 6 schematically shows an exemplary embodiment of the image acquisition apparatus
  • Fig. 7 schematically shows an exemplary embodiment of the workstation. Identical reference numerals are used to denote similar parts throughout the Figures. DETAILED DESCRIPTION OF EMBODIMENTS
  • Fig. 1 schematically shows a block diagram of an exemplary embodiment of the system SYS for visualizing an uncertainty band of a contour within a 2-dimensioanl (2-D) image, the system comprising:
  • a transfer unit U30 for applying an uncertainty transfer function, which assigns a renderable property to each location of a plurality of locations within the contour uncertainty band, based on the value of the uncertainty at said location;
  • a mixing unit U40 for computing a final image for displaying the 2-D image and the renderable property assigned to each location of the plurality of locations within the contour uncertainty band.
  • the exemplary embodiment of the system SYS further comprises: a contour unit U10 for computing the contour;
  • an uncertainty unit U20 for computing the uncertainty of the contour at each location of the plurality of locations within the contour uncertainty band
  • control unit U60 for controlling the work of the system SYS;
  • a memory unit U70 for storing data.
  • the first input connector U81 is arranged to receive data coming in from a data storage means such as, but not limited to, a hard disk, a magnetic tape, a flash memory, or an optical disk.
  • the second input connector U82 is arranged to receive data coming in from a user input device such as, but not limited to, a mouse or a touch screen.
  • the third input connector U83 is arranged to receive data coming in from a user input device such as a keyboard.
  • the input connectors U81, U82 and U83 are connected to an input control unit U80.
  • the first output connector U91 is arranged to output the data to a data storage means such as a hard disk, a magnetic tape, a flash memory, or an optical disk.
  • the second output connector U92 is arranged to output the data to a display device.
  • the output connectors U91 and U92 receive the respective data via an output control unit U90.
  • a person skilled in the art will understand that there are many ways to connect input devices to the input connectors U81, U82 and U83 and the output devices to the output connectors U91 and U92 of the system SYS. These ways comprise, but are not limited to, a wired and a wireless connection, a digital network such as, but not limited to, a Local Area Network (LAN) and a Wide Area Network (WAN), the Internet, a digital telephone network, and an analog telephone network.
  • LAN Local Area Network
  • WAN Wide Area Network
  • the system SYS comprises a memory unit U70.
  • the system SYS is arranged to receive input data from external devices via any of the input connectors U81, U82, and U83 and to store the received input data in the memory unit U70. Loading the input data into the memory unit U70 allows quick access to relevant data portions by the units of the system SYS.
  • the input data comprises image data and user settings for defining the uncertainty transfer function, for example.
  • the memory unit U70 may be implemented by devices such as, but not limited to, a register file of a CPU, a cache memory, a Random Access Memory (RAM) chip, a Read Only Memory (ROM) chip, and/or a hard disk drive and a hard disk.
  • the memory unit U70 may be further arranged to store the output data.
  • the output data comprises the image showing contour uncertainties within the contour uncertainty band.
  • the memory unit U70 may be also arranged to receive data from and/or deliver data to the units of the system SYS comprising the contour unit U10, the uncertainty unit U20, the transfer unit U30, the mixing unit U40, the control unit U60, and the user interface U65, via a memory bus U75.
  • the memory unit U70 is further arranged to make the output data available to external devices via any of the output connectors U91 and U92. Storing data from the units of the system SYS in the memory unit U70 may advantageously improve performance of the units of the system SYS as well as the rate of transfer of the output data from the units of the system SYS to external devices.
  • the system SYS comprises a control unit U60 for controlling the system SYS.
  • the control unit U60 may be arranged to receive control data from and provide control data to the units of the system SYS.
  • the uncertainty unit U20 may be arranged to provide control data "the uncertainties are computed" to the control unit U60, and the control unit U60 may be arranged to provide control data "use the uncertainty transfer function to compute the renderable property at the plurality of locations within the contour uncertainty band", to the transfer unit U30.
  • control functions may be implemented in other units of the system SYS.
  • the system SYS comprises a user interface U65 for enabling communication between a user and the system SYS.
  • the user interface U65 may be arranged to receive a user input comprising the name of the file comprising the 2-D image data.
  • the user interface may receive a user input for selecting a mode of operation of the system such as, for example, for selecting or modifying an uncertainty transfer function.
  • the user interface may be further arranged to display the final image.
  • the system SYS comprises the contour unit U10 for computing the contour.
  • the contour in a 2-D image can be computed using any method known in the art including, but not limited to, edge detection, region growing, level set approaches or model-based segmentation.
  • the system SYS is arranged for receiving or computing a plurality of locations within an uncertainty band B surrounding the contour.
  • the uncertainty unit U20 is arranged for computing the uncertainty of the contour at each location of the plurality of locations within the contour uncertainty band.
  • the uncertainty of the contour at a location (x, y) may be defined as the probability p(x, y) that the location (x, y) is on the contour.
  • the uncertainty may be defined as the probability 1 - p(x, y) that the location (x, y) does not lie on the contour.
  • the probability can be computed during segmentation. For example, let E(C) be an energy function of a contour C.
  • the energy function E(C) is minimized in the process of adapting the deformable contour C to the image using, for example, one of the methods described in A comparative study of deformable contour methods on medical image segmentation Lei He et al, Image and Vision Computing 26 (2008) 141-163, hereinafter referred to as Reference 1.
  • E m i n E(C m i n ) is the minimum energy for a certain contour C m i n adapted to the image as described in Reference 1.
  • C(x, y) be the contour adapted to the image, with the constraint that the location (x, y) of the plurality of locations within the contour uncertainty band is on the contour C(x, y) and let E(x, y) be the energy of the contour C(x, y).
  • E max Max ⁇ E(x, y): (x, y) e the plurality of locations within the uncertainty band ⁇ .
  • p(x, y)
  • the uncertainty unit U20 may be arranged to define a plurality of intervals Si, S 2 , ... perpendicular to the contour.
  • a number of locations rk,i, rk,2, ... e Sk can be selected.
  • the uncertainty at each location (x, y) of the plurality of locations within the contour uncertainty band is defined on the basis of the distance d(x, y) from said location to the contour and a gaussian probability density function g(x, y) c exp[-d(x, y) 2 /2o 2 ], wherein ⁇ 2 is the variance of the probability distribution defined by g(x, y).
  • the uncertainty of each location of the plurality of locations within the contour uncertainty band can be defined as an error of said location.
  • the error can be the radius of a circle centered at the location such that the probability that the circle comprises at least one location on the contour is greater than a predefined probability threshold.
  • the error can be a signed error.
  • one sign may correspond to the error at locations inside the area encircled by the contour while the other sign may correspond to the error at locations outside the area encircled by the contour.
  • For open contours one sign may correspond to the error at locations in the area on one side of the contour while the other sign may correspond to the error at locations in the area on the other side of the contour.
  • the plurality of locations within the contour uncertainty band may be substantially identical to the plurality of pixels substantially comprised in said uncertainty band.
  • the plurality of locations within the contour uncertainty band may comprise a subset of pixels substantially comprised in said uncertainty band.
  • Figure 2 shows two exemplary images showing an exemplary closed contour 22 and an exemplary open contour 24.
  • the uncertainties of each contour are visualized in the final images shown in Figures 3B and 4B, using two different uncertainty transfer functions for mapping the value of the uncertainty of the contour into a renderable property, wherein the two different uncertainty transfer functions are shown in Figures 3 A and 4A, respectively.
  • the first uncertainty transfer function is shown in Figure 3 A.
  • the RGB color coding is used to compute final images.
  • the green and blue channels of the final images shown in Figure 3B are identical to the green and blue channels of grayscale images shown in Figure 2.
  • Figure 4A shows two exemplary final images resulting from applying the second uncertainty function shown in Figure 4A to the 2-D image of Figure 2.
  • the error function shown in Figure 4A is discrete.
  • the mixing unit U40 of the system SYS is adapted for replacing the original grayscale values (I(x, y), I(x, y), I(x, y)) at each location (x, y) such that
  • 5 with the values (255, 0 , 0) corresponding to the red color, and for replacing the original grayscale values (I(x, y), I(x, y), I(x, y)) at each location (x, y) such that
  • 10 with the values (0, 0 , 255) corresponding to the blue color.
  • the contour uncertainty band comprises locations, wherein the uncertainty is not defined. These locations are not included in the plurality of locations comprised in the contour uncertainty band.
  • the system SYS may be arranged for computing uncertainties at additional locations in the contour uncertainty band on the basis of the uncertainties at the plurality of locations in the contour uncertainty band.
  • the skilled person will know various techniques suitable for computing the uncertainties at the additional locations, for example, interpolation techniques, or linear or nonlinear regression techniques.
  • system of the invention may be a valuable tool for assisting a physician in many aspects of her/his job.
  • embodiments of the system are illustrated using medical applications of the system, nonmedical applications of the system are also contemplated.
  • the units of the system SYS may be implemented using a processor.
  • Fig. 5 shows an exemplary flowchart of the method M of visualizing uncertainty of a contour within a contour uncertainty band in a 2-D image.
  • the method begins with a contour step S 10 for computing the contour.
  • the method M continues to an uncertainty step S20 for computing the uncertainty of the contour at each location of a plurality of locations within the contour uncertainty band.
  • the method M continues to an error transfer step S30 for applying the uncertainty transfer function, which assigns a renderable property to each location of the plurality of locations within the contour uncertainty band, based on the value of the uncertainty at said location.
  • the method M continues to a mixing step S40 for computing a final image for displaying the 2-D image and, at each location of the plurality of locations within the contour uncertainty band, the renderable property. After computing the final image, the method terminates.
  • a person skilled in the art may change the order of some steps, add some optional steps (e.g. user interaction for determining the uncertainty transfer function) or omit some non-mandatory steps, or perform some steps concurrently using threading models, multi-processor systems or multiple processes without departing from the concept as intended by the present invention.
  • two or more steps of the method M may be combined into one step.
  • a step of the method M may be split into a plurality of steps.
  • FIG. 6 schematically shows an exemplary embodiment of the image acquisition apparatus IAA employing the system SYS of the invention, said image acquisition apparatus IAA comprising an image acquisition unit IAA 10 connected via an internal connection with the system SYS, an input connector IAAOl, and an output connector IAA02.
  • This arrangement advantageously increases the capabilities of the image acquisition apparatus IAA, providing said image acquisition apparatus IAA with advantageous capabilities of the system SYS.
  • Fig. 7 schematically shows an exemplary embodiment of the workstation WS.
  • the workstation comprises a system bus WS01.
  • a processor WS10, a memory WS20, a disk input/output (I/O) adapter WS30, and a user interface WS40 are operatively connected to the system bus WS01.
  • a disk storage device WS31 is operatively coupled to the disk I/O adapter WS30.
  • a keyboard WS41, a mouse WS42, and a display WS43 are operatively coupled to the user interface WS40.
  • the system SYS of the invention implemented as a computer program, is stored in the disk storage device WS31.
  • the workstation WS00 is arranged to load the program and input data into memory WS20 and execute the program on the processor WS10.
  • the user can input information to the workstation WS00, using the keyboard WS41 and/or the mouse WS42.
  • the workstation is arranged to output information to the display device WS43 and/or to the disk WS31.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention relates to a system (SYS) for visualizing uncertainty of a contour within a contour uncertainty band in a 2-dimensional (2-D) image, the system comprising: a transfer unit (U30) for applying an uncertainty transfer function, which assigns a renderable property to each location of a plurality of locations within the contour uncertainty band, based on the value of the uncertainty at said location; and a mixing unit (U40) for computing a final image for displaying the 2-D image and the renderable property assigned to each location of the plurality of locations within the contour uncertainty band. The renderable property, e.g., a color, describes the uncertainty of the contour at each location of the plurality of locations within the contour uncertainty band. At each location of the plurality of locations within the contour uncertainty band, the uncertainty may be, for example, the probability that the location is (or, in an alternative embodiment, is not) on the contour. Since the final image comprises, at each location of the plurality of locations within the contour uncertainty band, both the 2-D image value and the renderable property, which can be displayed concurrently or in an alternating manner, the system of the invention is capable of showing the uncertainty of the contour without hiding image details.

Description

Visualization of uncertainty for 2-D lines
FIELD OF THE INVENTION:
The invention relates to visualizing uncertainties of object shapes plotted in 2-D images, especially contours computed from image data based on image segmentation. BACKGROUND OF THE INVENTION
A paper by Alex T. Pang et al. entitled "Approaches to Uncertainty
Visualization", published in The Visual Computer, Volume 13, Number 8, November 1997, describes techniques for visually presenting data together with their uncertainties. These uncertainty visualization techniques include adding glyphs, adding geometry, modifying geometry, modifying attributes, animation, sonification, and psycho-visual approaches, thus making the users aware of the magnitude of uncertainties present in their data.
The main shortcoming of current solutions is that the error visualization hides other information shown in the image behind the error plots. SUMMARY OF THE INVENTION
It would be advantageous to have a system, capable of showing the uncertainty of a contour in the image of interest, which allows seeing other image details.
Thus, in an aspect, the invention provides a system for visualizing the uncertainty of a contour within a contour uncertainty band in a 2-dimensioanl (2-D) image, the system comprising:
a transfer unit for applying an uncertainty transfer function, which assigns a renderable property to each location of a plurality of locations within the contour uncertainty band, based on the value of the uncertainty at said location; and
a mixing unit for computing a final image for displaying the 2-D image and the renderable property assigned to each location of the plurality of locations within the contour uncertainty band.
The renderable property, e.g., a color, describes the uncertainty of the contour at each location of the plurality of locations within the contour uncertainty band. At each location of the plurality of locations within the contour uncertainty band, the uncertainty may be, for example, the probability that the location is (or, in an alternative embodiment, is not) on the contour. Since the final image comprises, at each location of the plurality of locations within the contour uncertainty band, both the 2-D image value and the renderable property, which can be displayed concurrently or in an alternating manner, the system of the invention is capable of showing the uncertainty of the contour without hiding image details.
In an embodiment of the system, the mixing unit is arranged for determining a way of alternatingly displaying the 2-D image and the renderable property assigned by the uncertainty transfer function to each location of a plurality of locations within the contour uncertainty band. The final image is obtained by alternatingly displaying the 2-D image and an uncertainty image defined by the renderable property assigned by the uncertainty transfer function to each location of a plurality of locations within the contour uncertainty band. The display period for each image may be, for example, in the range of 0.1 to 1.0 second. This embodiment is particularly useful for illustrating uncertainties of temporal sequences of images.
In an embodiment of the system, the mixing unit is arranged for merging the
2-D image and the renderable property assigned by the uncertainty transfer function to each location of a plurality of locations within the contour uncertainty band. The merging may be accomplished by displaying the 2-D image in the red color channel R of the RGB color coding and an image defined by the renderable property assigned by the uncertainty transfer function to each location of a plurality of locations within the contour uncertainty band in the blue channel B of the RGB color coding, for example.
In an embodiment of the system, the renderable property comprises a color and a blending factor, and the mixing unit is arranged for blending the color assigned by the uncertainty transfer function to each location of the plurality of locations within the contour uncertainty band and the 2-D image using the blending factor assigned by the uncertainty transfer function to each location of the plurality of locations within the contour uncertainty band. Such blending has the effect that, at each location within the contour uncertainty band, the color indicates the contour uncertainty while the grayscale component shows image details, for example, an anatomical structure displayed in the image.
In an embodiment of the system, the uncertainty transfer function is a step function for visualizing a multiple contour uncertainty band using contours separating locations corresponding to a lower uncertainty from locations corresponding to a higher uncertainty. Such error function allows a clear quantitative visualization of a discrete contour uncertainty. In an embodiment, the system further comprises an uncertainty unit for computing the uncertainty of the contour at each location of the plurality of locations within the contour uncertainty band.
In an embodiment, the system further comprises a contour unit for computing the contour.
In an embodiment of the system, the 2-D image is a cross-section image computed from a 3 -dimensional (3-D) image, the contour is a cross-section of a surface identified in the 3-D image by the cross-section plane, and the uncertainty of the contour at each location of the plurality of locations within the contour uncertainty band is computed based on the uncertainty of the surface at each location of a plurality of locations in an uncertainty layer of the surface. This allows showing contour uncertainty in 2-D images derived from 3-D images.
In a further aspect, a workstation comprising the system of the invention is provided.
In a further aspect, an image acquisition apparatus comprising the system of the invention system is provided.
In a further aspect, a method of visualizing an uncertainty band of a contour within a 2-dimensioanl (2-D) image is provided, the method comprising:
an error transfer step for applying an uncertainty transfer function, which assigns a renderable property to each location of a plurality of locations within the contour uncertainty band, based on the value of the uncertainty at said location; and
a mixing step for computing a final image for displaying the 2-D image and the renderable property assigned to each location of the plurality of locations within the contour uncertainty band.
In an implementation, the method further comprises an uncertainty step for computing the uncertainty of the contour at each location of the plurality of locations within the contour uncertainty band.
In an implementation, the method further comprises a contour step for computing the contour.
In a further aspect, the invention provides a computer program product to be loaded by a computer arrangement, the computer program product comprising instructions for visualizing an uncertainty band of a contour within a 2-dimensioanl (2-D) image, the computer arrangement comprising a processing unit and a memory, the computer program product, after being loaded, providing said processing unit with the capability to carry out steps of the method of the invention.
It will be appreciated by those skilled in the art that two or more of the above- mentioned embodiments, implementations, and/or aspects of the invention may be combined in any way deemed useful.
Modifications and variations of the system, of the workstation, of the image acquisition apparatus, of the method, and/or of the computer program product, which correspond to the described modifications and variations of the system or of the method, can be carried out by a person skilled in the art on the basis of the description.
The invention is defined in the independent claims. Advantageous embodiments are defined in the dependent claims.
BRIEF DESCRIPTION OF THE DRAWINGS
These and other aspects of the invention are apparent from and will be elucidated by means of implementations and embodiments described hereinafter and with reference to the accompanying drawings, wherein:
Fig. 1 shows a block diagram of an exemplary embodiment of the system; Fig. 2 shows two exemplary images, each showing an exemplary contour; Fig. 3A shows a first exemplary uncertainty transfer function;
Fig. 3B shows two exemplary final images resulting from applying the first uncertainty function shown in Fig. 3 A to the images of Fig. 2;
Fig. 4A shows a second exemplary uncertainty transfer function; Fig. 4B shows two exemplary final images resulting from applying the second uncertainty function shown in Fig. 4A to the images of Fig. 2;
Fig. 5 schematically shows an exemplary flowchart of the method;
Fig. 6 schematically shows an exemplary embodiment of the image acquisition apparatus; and
Fig. 7 schematically shows an exemplary embodiment of the workstation. Identical reference numerals are used to denote similar parts throughout the Figures. DETAILED DESCRIPTION OF EMBODIMENTS
Fig. 1 schematically shows a block diagram of an exemplary embodiment of the system SYS for visualizing an uncertainty band of a contour within a 2-dimensioanl (2-D) image, the system comprising:
- a transfer unit U30 for applying an uncertainty transfer function, which assigns a renderable property to each location of a plurality of locations within the contour uncertainty band, based on the value of the uncertainty at said location; and
a mixing unit U40 for computing a final image for displaying the 2-D image and the renderable property assigned to each location of the plurality of locations within the contour uncertainty band.
The exemplary embodiment of the system SYS further comprises: a contour unit U10 for computing the contour;
an uncertainty unit U20 for computing the uncertainty of the contour at each location of the plurality of locations within the contour uncertainty band;
- a control unit U60 for controlling the work of the system SYS;
a user interface U65 for communication between the user and the system SYS; and
a memory unit U70 for storing data.
In an embodiment of the system SYS, there are three input connectors U81, U82 and U83 for the incoming data. The first input connector U81 is arranged to receive data coming in from a data storage means such as, but not limited to, a hard disk, a magnetic tape, a flash memory, or an optical disk. The second input connector U82 is arranged to receive data coming in from a user input device such as, but not limited to, a mouse or a touch screen. The third input connector U83 is arranged to receive data coming in from a user input device such as a keyboard. The input connectors U81, U82 and U83 are connected to an input control unit U80.
In an embodiment of the system SYS, there are two output connectors U91 and U92 for the outgoing data. The first output connector U91 is arranged to output the data to a data storage means such as a hard disk, a magnetic tape, a flash memory, or an optical disk. The second output connector U92 is arranged to output the data to a display device. The output connectors U91 and U92 receive the respective data via an output control unit U90.
A person skilled in the art will understand that there are many ways to connect input devices to the input connectors U81, U82 and U83 and the output devices to the output connectors U91 and U92 of the system SYS. These ways comprise, but are not limited to, a wired and a wireless connection, a digital network such as, but not limited to, a Local Area Network (LAN) and a Wide Area Network (WAN), the Internet, a digital telephone network, and an analog telephone network.
In an embodiment, the system SYS comprises a memory unit U70. The system SYS is arranged to receive input data from external devices via any of the input connectors U81, U82, and U83 and to store the received input data in the memory unit U70. Loading the input data into the memory unit U70 allows quick access to relevant data portions by the units of the system SYS. The input data comprises image data and user settings for defining the uncertainty transfer function, for example. The memory unit U70 may be implemented by devices such as, but not limited to, a register file of a CPU, a cache memory, a Random Access Memory (RAM) chip, a Read Only Memory (ROM) chip, and/or a hard disk drive and a hard disk. The memory unit U70 may be further arranged to store the output data. The output data comprises the image showing contour uncertainties within the contour uncertainty band. The memory unit U70 may be also arranged to receive data from and/or deliver data to the units of the system SYS comprising the contour unit U10, the uncertainty unit U20, the transfer unit U30, the mixing unit U40, the control unit U60, and the user interface U65, via a memory bus U75. The memory unit U70 is further arranged to make the output data available to external devices via any of the output connectors U91 and U92. Storing data from the units of the system SYS in the memory unit U70 may advantageously improve performance of the units of the system SYS as well as the rate of transfer of the output data from the units of the system SYS to external devices.
In an embodiment, the system SYS comprises a control unit U60 for controlling the system SYS. The control unit U60 may be arranged to receive control data from and provide control data to the units of the system SYS. For example, after computing the uncertainties at the plurality of locations within the contour uncertainty band, the uncertainty unit U20 may be arranged to provide control data "the uncertainties are computed" to the control unit U60, and the control unit U60 may be arranged to provide control data "use the uncertainty transfer function to compute the renderable property at the plurality of locations within the contour uncertainty band", to the transfer unit U30.
Alternatively, control functions may be implemented in other units of the system SYS.
In an embodiment of the system SYS, the system SYS comprises a user interface U65 for enabling communication between a user and the system SYS. The user interface U65 may be arranged to receive a user input comprising the name of the file comprising the 2-D image data. Optionally, the user interface may receive a user input for selecting a mode of operation of the system such as, for example, for selecting or modifying an uncertainty transfer function. The user interface may be further arranged to display the final image. A person skilled in the art will understand that more functions may be advantageously implemented in the user interface U65 of the system SYS.
In an embodiment, the system SYS comprises the contour unit U10 for computing the contour. The contour in a 2-D image can be computed using any method known in the art including, but not limited to, edge detection, region growing, level set approaches or model-based segmentation.
In an embodiment, the system SYS is arranged for receiving or computing a plurality of locations within an uncertainty band B surrounding the contour. Then, the uncertainty unit U20 is arranged for computing the uncertainty of the contour at each location of the plurality of locations within the contour uncertainty band. The uncertainty of the contour at a location (x, y) may be defined as the probability p(x, y) that the location (x, y) is on the contour. Alternatively, the uncertainty may be defined as the probability 1 - p(x, y) that the location (x, y) does not lie on the contour. The probability can be computed during segmentation. For example, let E(C) be an energy function of a contour C. The energy function E(C) is minimized in the process of adapting the deformable contour C to the image using, for example, one of the methods described in A comparative study of deformable contour methods on medical image segmentation Lei He et al, Image and Vision Computing 26 (2008) 141-163, hereinafter referred to as Reference 1. Emin = E(Cmin) is the minimum energy for a certain contour Cmin adapted to the image as described in Reference 1. Let C(x, y) be the contour adapted to the image, with the constraint that the location (x, y) of the plurality of locations within the contour uncertainty band is on the contour C(x, y) and let E(x, y) be the energy of the contour C(x, y). Let Emax = Max{E(x, y): (x, y) e the plurality of locations within the uncertainty band} . Then p(x, y) =| Emax - E(x, y)|/|Emax - Emin| is a measure of the probability that (x, y) is on the contour.
In another embodiment of the system SYS, the uncertainty unit U20 may be arranged to define a plurality of intervals Si, S2, ... perpendicular to the contour. On each interval Sk, a number of locations rk,i, rk,2, ... e Sk can be selected. The set { rkj: k = 1, 2, ... and j = 1, 2, ...} defines the plurality of locations within the contour uncertainty band. On the basis of the intensity profile I(r), r e Sk, the uncertainty unit U20 can be further adapted for computing the probabilities ρ(¾), wherein∑k= ι, 2, ... p(¾,j) = 1, such that the location ¾j lies on the contour. In an embodiment of the system SYS, the uncertainty at each location (x, y) of the plurality of locations within the contour uncertainty band is defined on the basis of the distance d(x, y) from said location to the contour and a gaussian probability density function g(x, y) c exp[-d(x, y)2/2o2], wherein σ2 is the variance of the probability distribution defined by g(x, y).
Optionally, the uncertainty of each location of the plurality of locations within the contour uncertainty band can be defined as an error of said location. For example, the error can be the radius of a circle centered at the location such that the probability that the circle comprises at least one location on the contour is greater than a predefined probability threshold. Optionally, the error can be a signed error. For closed contours, one sign may correspond to the error at locations inside the area encircled by the contour while the other sign may correspond to the error at locations outside the area encircled by the contour. For open contours, one sign may correspond to the error at locations in the area on one side of the contour while the other sign may correspond to the error at locations in the area on the other side of the contour.
The plurality of locations within the contour uncertainty band may be substantially identical to the plurality of pixels substantially comprised in said uncertainty band. Alternatively, the plurality of locations within the contour uncertainty band may comprise a subset of pixels substantially comprised in said uncertainty band.
The skilled person will know many methods of defining the plurality of locations within the contour uncertainty band and computing uncertainties . The scope of the claims should not be construed as being limited to using any specific method.
Figure 2 shows two exemplary images showing an exemplary closed contour 22 and an exemplary open contour 24. The uncertainties of each contour are visualized in the final images shown in Figures 3B and 4B, using two different uncertainty transfer functions for mapping the value of the uncertainty of the contour into a renderable property, wherein the two different uncertainty transfer functions are shown in Figures 3 A and 4A, respectively.
The first uncertainty transfer function is shown in Figure 3 A. The function assigns a blending coefficient a: a(x, y) = 1 - |δ(χ, y)|/10 to an error δ(χ, y) at locations (x, y) such that |δ(χ, y)| < 10. The RGB color coding is used to compute final images. The red channel Rl(x, y) of the final images shown in Figure 3B is computed from the red channel R(x, y) of grayscale images shown in Figure 2, using the formula Rl(x, y) = (1- a(x, y))*R(x, y) + a(x, y)*R0, where R0 is a predefined constant. The green and blue channels of the final images shown in Figure 3B are identical to the green and blue channels of grayscale images shown in Figure 2.
Figure 4A shows two exemplary final images resulting from applying the second uncertainty function shown in Figure 4A to the 2-D image of Figure 2. The error function shown in Figure 4A is discrete. Using this error function, the mixing unit U40 of the system SYS is adapted for replacing the original grayscale values (I(x, y), I(x, y), I(x, y)) at each location (x, y) such that |δ(χ, y)| = 5 with the values (255, 0 , 0) corresponding to the red color, and for replacing the original grayscale values (I(x, y), I(x, y), I(x, y)) at each location (x, y) such that |δ(χ, y)| = 10 with the values (0, 0 , 255) corresponding to the blue color.
In an embodiment of the system SYS, the contour uncertainty band comprises locations, wherein the uncertainty is not defined. These locations are not included in the plurality of locations comprised in the contour uncertainty band. In such a case, the system SYS may be arranged for computing uncertainties at additional locations in the contour uncertainty band on the basis of the uncertainties at the plurality of locations in the contour uncertainty band. The skilled person will know various techniques suitable for computing the uncertainties at the additional locations, for example, interpolation techniques, or linear or nonlinear regression techniques.
A person skilled in the art will appreciate that the system of the invention may be a valuable tool for assisting a physician in many aspects of her/his job. Further, although the embodiments of the system are illustrated using medical applications of the system, nonmedical applications of the system are also contemplated.
Those skilled in the art will further understand that other embodiments of the system SYS are also possible. It is possible, among other things, to redefine the units of the system and to redistribute their functions. Although the described embodiments apply to medical images, other applications of the system, not related to medical applications, are also possible.
The units of the system SYS may be implemented using a processor.
Normally, their functions are performed under the control of a software program product. During execution, the software program product is normally loaded into a memory, like a RAM, and executed from there. The program may be loaded from a background memory, such as a ROM, hard disk, or magnetic and/or optical storage, or may be loaded via a network like the Internet. Optionally, an application-specific integrated circuit may provide the described functionality. Fig. 5 shows an exemplary flowchart of the method M of visualizing uncertainty of a contour within a contour uncertainty band in a 2-D image. In an
implementation, the method begins with a contour step S 10 for computing the contour. After the contour step S10, the method M continues to an uncertainty step S20 for computing the uncertainty of the contour at each location of a plurality of locations within the contour uncertainty band. After computing the uncertainties, the method M continues to an error transfer step S30 for applying the uncertainty transfer function, which assigns a renderable property to each location of the plurality of locations within the contour uncertainty band, based on the value of the uncertainty at said location. After the error transfer step S30, the method M continues to a mixing step S40 for computing a final image for displaying the 2-D image and, at each location of the plurality of locations within the contour uncertainty band, the renderable property. After computing the final image, the method terminates.
A person skilled in the art may change the order of some steps, add some optional steps (e.g. user interaction for determining the uncertainty transfer function) or omit some non-mandatory steps, or perform some steps concurrently using threading models, multi-processor systems or multiple processes without departing from the concept as intended by the present invention. Optionally, two or more steps of the method M may be combined into one step. Optionally, a step of the method M may be split into a plurality of steps.
Fig. 6 schematically shows an exemplary embodiment of the image acquisition apparatus IAA employing the system SYS of the invention, said image acquisition apparatus IAA comprising an image acquisition unit IAA 10 connected via an internal connection with the system SYS, an input connector IAAOl, and an output connector IAA02. This arrangement advantageously increases the capabilities of the image acquisition apparatus IAA, providing said image acquisition apparatus IAA with advantageous capabilities of the system SYS.
Fig. 7 schematically shows an exemplary embodiment of the workstation WS. The workstation comprises a system bus WS01. A processor WS10, a memory WS20, a disk input/output (I/O) adapter WS30, and a user interface WS40 are operatively connected to the system bus WS01. A disk storage device WS31 is operatively coupled to the disk I/O adapter WS30. A keyboard WS41, a mouse WS42, and a display WS43 are operatively coupled to the user interface WS40. The system SYS of the invention, implemented as a computer program, is stored in the disk storage device WS31. The workstation WS00 is arranged to load the program and input data into memory WS20 and execute the program on the processor WS10. The user can input information to the workstation WS00, using the keyboard WS41 and/or the mouse WS42. The workstation is arranged to output information to the display device WS43 and/or to the disk WS31. A person skilled in the art will understand that there are numerous other embodiments of the workstation WS known in the art and that the present embodiment serves the purpose of illustrating the invention and must not be interpreted as limiting the invention to this particular embodiment.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim or in the description. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements and by means of a programmed computer. In the system claims enumerating several units, several of these units can be embodied by one and the same record of hardware or software. The usage of the words first, second, third, etc., does not indicate any ordering. These words are to be interpreted as names.

Claims

CLAIMS:
1. A system (SYS) for visualizing uncertainty of a contour within a contour uncertainty band in a 2-dimensioanl (2-D) image, the system comprising:
a transfer unit (U30) for applying an uncertainty transfer function, which assigns a renderable property to each location of a plurality of locations within the contour uncertainty band, based on the value of the uncertainty at said location; and
a mixing unit (U40) for computing a final image for displaying the 2-D image and the renderable property assigned to each location of the plurality of locations within the contour uncertainty band.
2. The system (SYS) as claimed in claim 1, wherein the mixing unit (U40) is adapted for alternatingly displaying the 2-D image and the renderable property assigned to each location of the plurality of locations within the contour uncertainty band.
3. The system (SYS) as claimed in claim 1, wherein the mixing unit (U40) is adapted for merging the reference image and the renderable property assigned to each location of the plurality of locations within the contour uncertainty band.
4. The system (SYS) as claimed in claim 3, wherein the renderable property comprises a color and a blending factor, and wherein the mixing unit (U40) is arranged for blending the color assigned by the uncertainty transfer function to each location of the plurality of locations within the contour uncertainty band and the 2-D image using the blending factor assigned by the uncertainty transfer function to each location of the plurality of locations within the contour uncertainty band.
5. The system (SYS) as claimed in claim 1, wherein the uncertainty transfer function is a step function for visualizing a multiple contour uncertainty band using contours separating locations corresponding to a lower uncertainty from locations corresponding to a higher uncertainty.
6. The system (SYS) as claimed in claim 1, further comprising an uncertainty unit (U20) for computing the uncertainty of the contour at each location of the plurality of locations within the contour uncertainty band.
7. The system (SYS) as claimed in claim 1, further comprising a contour unit
(U10) for computing the contour.
8. The system (SYS) as claimed in claim 6, wherein the 2-D image is a cross- section image computed from a 3 -dimensional (3-D) image, the contour is a cross-section of a surface identified in the 3-D image by the cross-section plane, and the uncertainty of the contour at each location of the plurality of locations within the contour uncertainty band is computed based on the uncertainty of the surface at each location of a plurality of locations within an uncertainty layer of the surface.
9. A workstation (WS) comprising a system (SYS) as claimed in any one of the previous claims.
10. An image acquisition apparatus (IAA) comprising a system (SYS) as claimed in any one of the previous claims.
11. A method (M) of visualizing uncertainty of a contour within a contour uncertainty band in a 2-dimensioanl (2-D) image, the method comprising:
an error transfer step (S30) for applying an uncertainty transfer function, which assigns a renderable property to each location of a plurality of locations within the contour uncertainty band, based on the value of the uncertainty at said location; and
a mixing step (S40) for computing a final image for displaying the 2-D image and the renderable property assigned to each location of the plurality of locations within the contour uncertainty band.
12. The method (M) as claimed in claim 11, further comprising an uncertainty step
(S20) for computing the uncertainty of the contour at each location of the plurality of locations within the contour uncertainty band.
13. The method (M) as claimed in claim 11, further comprising a contour step (S10) for computing the contour.
14. A computer program product to be loaded by a computer arrangement, comprising instructions for visualizing uncertainty of a contour within a contour uncertainty band in a 2-dimensioanl (2-D) image, the computer arrangement comprising a processing unit and a memory, the computer program product, after being loaded, providing said processing unit with the capability to carry out steps of a method as claimed in claim 11, 12 or 13.
PCT/IB2011/055661 2010-12-22 2011-12-14 Visualization of uncertainty for 2-d lines WO2012085765A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP10196406 2010-12-22
EP10196406.2 2010-12-22

Publications (2)

Publication Number Publication Date
WO2012085765A2 true WO2012085765A2 (en) 2012-06-28
WO2012085765A3 WO2012085765A3 (en) 2012-08-16

Family

ID=45470613

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2011/055661 WO2012085765A2 (en) 2010-12-22 2011-12-14 Visualization of uncertainty for 2-d lines

Country Status (1)

Country Link
WO (1) WO2012085765A2 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10592743B2 (en) 2017-08-24 2020-03-17 International Business Machines Corporation Machine learning to predict cognitive image composition

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
ALEX T. PANG ET AL.: "Approaches to Uncertainty Visualization", THE VISUAL COMPUTER, vol. 13, no. 8, November 1997 (1997-11-01), XP055028661, DOI: doi:10.1007/s003710050111
LEI HE ET AL., IMAGE AND VISION COMPUTING, vol. 26, 2008, pages 141 - 163

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10592743B2 (en) 2017-08-24 2020-03-17 International Business Machines Corporation Machine learning to predict cognitive image composition
US11468664B2 (en) 2017-08-24 2022-10-11 International Business Machines Corporation Machine learning to predict cognitive image composition

Also Published As

Publication number Publication date
WO2012085765A3 (en) 2012-08-16

Similar Documents

Publication Publication Date Title
WO2021057848A1 (en) Network training method, image processing method, network, terminal device and medium
JP6636678B2 (en) Learning to annotate objects in images
US10073602B2 (en) System and method for displaying a suggested luminance adjustment for an image
US8248410B2 (en) Synthesizing detailed depth maps from images
US11776243B2 (en) Systems and methods for automated detection of visual objects in medical images
CN110136056A (en) The method and apparatus of image super-resolution rebuilding
US8401285B1 (en) Methods, apparatuses, and computer program products for controlling luminance of non-tissue objects within an image
EP1555634A1 (en) Method and apparatus for automatically determining image foreground color
CN107622504B (en) Method and device for processing pictures
US11683438B2 (en) Systems and methods to semi-automatically segment a 3D medical image using a real-time edge-aware brush
US8605973B2 (en) Graph cuts-based interactive segmentation of teeth in 3-D CT volumetric data
CN116109824A (en) Medical image and pixel-level label generation method and device based on diffusion model
EP2485190A1 (en) Adapting the resolution of a graphic representation of metadata
JP2012510641A (en) Medical image viewer
EP2659457B1 (en) Tnm classification using image overlays
JP5872579B2 (en) Image processing device
US9727999B2 (en) Visualization of flow patterns
JP2018185265A (en) Information processor, method for control, and program
KR20160018675A (en) Volume rendering of images with multiple classifications
US9552531B2 (en) Fast color-brightness-based methods for image segmentation
AU2012268887A1 (en) Saliency prediction method
WO2012085765A2 (en) Visualization of uncertainty for 2-d lines
CN106530386B (en) Method and system for volume rendering of medical images
US11727629B2 (en) Un-occluded visualization of focus with depth continuity with context in volume rendering
Hwang Automatic information loss detection and color compensation for the color blind

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11807770

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11807770

Country of ref document: EP

Kind code of ref document: A2