US20090003651A1

US20090003651A1 - Object segmentation recognition

Info

Publication number: US20090003651A1
Application number: US12/129,036
Authority: US
Inventors: Peter Dugan; Kenei Suntarat; Robert L. Finch; Rosemary D. Paradis
Original assignee: Lockheed Martin Corp
Current assignee: Lockheed Martin Corp
Priority date: 2007-05-29
Filing date: 2008-05-29
Publication date: 2009-01-01
Also published as: US20090052762A1; US8094874B2; US20090003699A1; US20090052622A1; US20090052732A1; US20090055344A1; US20080298544A1

Abstract

A system for segmenting radiographic images of a cargo container can include an object segmentation recognition module adapted to perform a series of functions. The functions can include receiving a plurality of radiographic images of a cargo container, each image generated using a different energy level and segmenting each of the radiographic images using one or more segmentation modules to generate segmentation data representing one or more image segments. The functions can also include identifying image layers within the radiographic images using a plurality of layer analysis modules by providing the plurality of radiographic images and the segmentation data as input to the layer analysis modules, and determining adjusted atomic number values for an atomic number image based on the image layers. The functions can include adjusting the atomic number image based on the adjusted atomic number values for the regions of interest to generate an adjusted atomic number image and identifying regions of interest within the adjusted atomic number image based on an image characteristic. The functions can also include providing coordinates of each region of interest and the adjusted atomic number image as output.

Description

The present application claims the benefit of U.S. Provisional Patent Application No. 60/940,632, entitled “Threat Detection System”, filed May 29, 2007, which is incorporated herein by reference in its entirety.
Embodiments of the present invention relate generally to image segmentation and, more particularly, to computer systems and methods for automatic image segmentation of radiographic images.
Image segmentation, the process of separating objects of interest from the background (or from other objects) in an image, is typically a difficult task for a computer to perform. If an image scene is simple and the contrast between objects in the scene and the background is high, then the task may be somewhat easier. However, if an image scene is cluttered and the contrast between objects in the scene and the background (or other objects) is low, image segmentation can be a particularly difficult problem. For example, in a radiographic image of a three-dimensional object such as a cargo container there can be numerous layers of objects and contrast may be low between the objects and the background. In addition to the difficulties often associated with low contrast and cluttered scenes, radiographic images of objects having layers may also present a need to segment the image in two ways: in the x-y plane (i.e., the plane the image was produced on) and by layer of depth in order to correct for layer effects such as overlapping.
Embodiments of the present invention can be used in an imaging system, such as a nuclear material detection system, that includes a capability of producing images using different energy levels. Each energy level provides a different imaging characteristic such as energy penetration of the object being scanned. Different images produced using different energy levels can be used in conjunction with each other to better identify layers within the object being scanned.
Embodiments of the present invention address the above problems and other problems often associated with segmenting radiographic images. For example, one exemplary embodiment can include a system for segmenting radiographic images of a cargo container. The system can include an object segmentation recognition module adapted to perform a series of functions. The functions can include receiving a plurality of radiographic images of a cargo container, each image generated using a different energy level and segmenting each of the radiographic images using one or more segmentation modules to generate segmentation data representing one or more image segments. The functions can also include identifying image layers within the radiographic images using a plurality of layer analysis modules by providing the plurality of radiographic images and the segmentation data as input to the layer analysis modules, and determining adjusted atomic number values for an atomic number image based on the image layers. The functions can include adjusting the atomic number image based on the adjusted atomic number values for the regions of interest to generate an adjusted atomic number image and identifying regions of interest within the adjusted atomic number image based on an image characteristic. The functions can also include providing coordinates of each region of interest and the adjusted atomic number image as output.
Another embodiment includes a method for segmenting radiographic images. The method can include providing a plurality of radiographic images and providing a pixel level estimated atomic number image. The method can also include segmenting the radiographic images using one or more segmentation modules to generate segmentation data and identifying image layers within each of the radiographic images. The method can also include analyzing the image layers using the plurality of radiographic images and the segmentation data in order to determine corrected atomic number values for objects in the images, and correcting the pixel level atomic number image based on the corrected atomic number values to generate an object level atomic number image. The method can include identifying regions of interest within the object level atomic number image based on an image characteristic, and outputting coordinates of each region of interest and the object level atomic number image as output.
Another embodiment includes a radiographic image segmentation apparatus. The apparatus can include means for receiving a radiographic image and means for segmenting each of the radiographic images using one or more segmentation modules to generate segmentation data. The apparatus can also include means for identifying image layers to produce image layer data and means for identifying regions of interest based on the segmentation data and image layer data. The apparatus can also include means for analyzing the image layers in order to determine corrected atomic number values for the regions of interest and means for correcting an atomic number image based on the corrected atomic number values for the regions of interest. The apparatus can also include means for outputting region of interest coordinates and the corrected atomic number image as output.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an exemplary object segmentation recognition processor showing inputs and outputs;

FIG. 2 is a block diagram of an exemplary object segmentation recognition processor showing an exemplary OSR processor in detail;

FIG. 3 is a flowchart showing an exemplary method for image segmentation; and

FIG. 4 is a block diagram of an exemplary object segmentation recognition apparatus showing data flow and processing modules.

DETAILED DESCRIPTION

FIG. 1 shows a block diagram of an exemplary object segmentation recognition processor showing inputs and outputs. In particular, an object segmentation recognition (OSR) processor 102 is shown receiving one or more images 104 as input and providing region of interest (ROI) or object coordinates 106 as output.
In operation, the images 104 provided or obtained as input to the OSR processor 102 can include radiographic images or other images. For example, the images 104 can include radiographic images of a cargo conveyance such as a cargo container. The images 104 can include one or more images, for example four images can be provided with each image being generated using a different radiographic energy level. Also, the images 104 can include radiographic images or other images derived from radiographic images, such as, for example, an atomic number image representing estimated atomic numbers associated with radiographic images.
The OSR processor 102 can obtain, request or receive the images 104 via a wired or wireless connection, such as a network (e.g., LAN, WAN, wireless network, Internet or the like) or direct connection within a system. The OSR processor 102 can also receive the images 104 via a software connection (e.g., procedure call, standard object access protocol, remote procedure call, or the like). In general, any known or later developed wired, wireless or software connection suitable for transmitting data can be used to supply the images 104 to the OSR processor 102. The OSR processor 102 can be requested to segment images by another process or system, or can request images for segmenting from another process or system. If the images 104 include more than one image, the images can be registered prior to being sent for segmentation.
The OSR processor 102 processes the images 104 to segment the images 104 and identify objects within the images 104. The OSR processor 102 can also extract or identify layers (or estimated layers) within the images in order to help segment the images more accurately. The layer information can also be used to correct or adjust estimated atomic numbers in an atomic number image or map. The atomic number image or map can include a representation of estimated atomic numbers determined from the images 104.
Once the images 104 have been segmented and the layer information has been determined, regions of interest (ROIs) within the images 104 can be located or determined. The ROIs can be determined based on an image characteristic such as estimated atomic number of the ROI (or object), shape of the ROI, position or location of the ROI, or the like. The OSR processor 102 can provide ROI/object coordinates 106 as output. The ROI/object coordinates 106 can be associated with the input images 104 or an atomic number image. The output ROI/object coordinates 106 can be outputted via a wired or wireless connection, such as a network (e.g., LAN, WAN, Internet or the like) or direct connection within a system. The output ROI/object coordinates 106 can be outputted via a software connection (e.g., response to a procedure call, standard object access protocol, remote procedure call, or the like).
FIG. 2 is a block diagram of an exemplary object segmentation recognition processor showing an exemplary OSR processor in detail. In addition to the components already described above, the OSR processor 102 includes a segment processing section 202 having a connected region analysis module 204, an edge analysis module 206, a ratio layer analysis module 208 and a blind source separation (BSS) layer analysis module 210. The OSR processor 102 also includes an object ROI section 212 having a layer analysis and segment association module 214 and an object ROI determination module 216.
In operation, the segment processing section receives the images 104. Once received, the images 104 can be processed using one or more image segmentation modules (e.g., the connected region analysis module 204, the edge analysis module 206, or a combination of the above). It will be appreciated that the segmentation modules shown are for illustration purposes and that any known or later developed image segmentation processes can be used. Also, the selection of the number and type of image segmentation modules employed in the OSR processor 102 may depend on a contemplated use of an embodiment and the selection may be guided by a number of factors including, but not limited to, type of materials being scanned, configuration of the scanning system and objects being scanned, desired performance characteristics, time available for processing, or the like. The individual segmentation processes, layer analysis processes, or both may be performed sequentially or in parallel or a combination of the above and in any suitable order.
Once the segmentation processing (object segmentation, layer analysis, or both) has been completed, the resulting image segment data can be provided to the object ROI section 212. In the object ROI section 212, the layers and segments of the image segment data are analyzed using layer analysis and segment association module 214 and combined or associated to produce segment-layer data that contains information about objects and layers within the images 104. The images 104 can be processed by one or more layer analysis modules (e.g., the ratio layer analysis module 208, the BSS layer analysis module 210, or a combination of the above) within the layer analysis and segment association module 214.
The segment-layer data can be in the form of an atomic number image that represents a composite of the images 104 and has been adjusted or corrected based on layers and segments to provide an image suitable for identification of ROIs. The segment-layer data can also be represented in any form suitable for transmitting the information that may be needed to analyze the images 104. The segment-layer data is then provided to the object ROI determination module 216 for analysis and identification of ROIs.
The object ROI determination module 216 can use one or more image characteristics to identify ROIs within the images 104 or the segment-layer data. Image characteristics can include an estimated atomic number for a portion of the image (e.g., a pixel, segment, object, region or the like), a shape of a segment or object within the image, or a position or location of an object or segment. In general, any image characteristic that is suitable for identifying an ROI can be used.
Once the ROIs have been determined, coordinate data (106) representing each ROI can be provided as output. The output can be provided as described above in connection with reference number 106 of FIG. 1. Also, segment-layer data or an adjusted or corrected atomic number image can be provided in addition to, or as a substitute for, the ROI coordinates.
FIG. 3 is a flowchart showing an exemplary computer implemented method for image segmentation. Processing begins at step 302 and continues to step 304.
In step 304, one or more radiographic images are obtained. These images can be provided by an imaging system (e.g., an x-ray, magnetic resonance imaging device, computerized tomography device, or the like). In general, any imaging device suitable for generating images that may require segmenting can be used. Processing continues to step 306.
In step 306, the radiographic images are segmented. The segmentation can be performed using one or more image segmentation processes. Examples of segmentation methods include modules or processes for segmentation based on clustering, histograms, edge detection, region growing, level set, graph partitioning, watershed, model based, and multi-scale. Processing continues to step 308.
In step 308, any layers present in the images are determined. The layers can be determined using one or more layer extraction or identification processes. For example, a ratio layer analysis process and a BSS layer analysis process can be used together to identify layers in the images. For example, using the layer analysis module 214 mentioned above. A goal of layer identification and extraction is to remove overlapping effects which may be present. By removing overlapping effects, the true gray level of a material can be determined. Using the true gray level, a material's effective atomic number (and, optionally, material density) can be determined. Using the effective atomic number the composition of the material can help in determining illicit materials, such as special nuclear materials can be detected automatically.
The ratio method of layer identification and overlap effect removal is known in the art as applied to dual energy and is described in “The Utility of X-ray Dual-energy Transmission and Scatter Technologies for Illicit Material Detection,” a published Ph.D. Dissertation by Lu Qiang, Virginia Polytechnic Institute and State University, Blacksburg, Va., 1999, which is incorporated herein by reference in its entirety. Generally, the ratio method provides a process whereby a computer can solve for one image layer and remove any overlapping effects of another layer. Thus, regions that overlap or may be obscured can be separated into their constituent layers and a true gray level can be determined for each layer. The true gray level can be used to more accurately determine an estimated atomic number an/or material type.
Blind source separation (or blind signal separation) is a technique known in the art, and refers generally to the separation of a set of signals from a set of mixed signals, without the aid of information (or with very little information) about the source signals or the mixing process. However, if information about how the signals were mixed (e.g., a mixing matrix) can be estimated, it can then be used to determine an un-mixing matrix which can be applied to separate the components within a mixed signal.
The BSS method may be limited by the amount of independence between materials within the mixture. Several techniques exist for estimating the mixing matrix, some include using an unsupervised learning process. The process can include incrementally changing and weighting coefficients of the mixing matrix and evaluating the mixing matrix until optimal conditions are met. Once the mixing matrix is estimated, un-mixing coefficients can be computed. Examples of some BSS techniques include projection pursuit gradient ascent, projection pursuit stepwise separation, ICA gradient ascent, and complexity pursuit gradient ascent. In general, an iterative hill climbing or other type of optimization process can be used to estimate the mixing matrix and determine an optimal matrix. Also, contemplated or desired performance levels may require development of custom algorithms that can be tuned to a specific empirical terrain provided by the mixing and un-mixing matrices. Once the layers are identified and overlapping effects are removed, processing continues to step 310.
In step 310, segments that have been identified are associated with any layers that have been determined or identified in step 308. Associating segments with layers can help to remove any overlapping effects and also can improve the ability to determine a true gray value for a segment. Processing continues to step 312.
In step 312, ROIs are determined. The ROIs can be determined based on an image characteristic as described above. Processing continues to step 314.
In step 314, a gray level atomic number image is optionally adjusted to reflect the corrections or adjustments provided by the layer determination. The adjustments or corrections can include changes related to removal of overlap effects or other changes. Processing continues to step 316.
In step 316, the ROI coordinates and, optionally, the adjusted or corrected gray level image are provided as output to an operator or another system. The output can be in a standard format or in a proprietary format. Processing continues to step 318 where the method ends. It will be appreciated that steps 304-316 can be repeated in whole or in part to perform a contemplated image segmentation process.
FIG. 4 is a block diagram of an exemplary object segmentation recognition apparatus showing data flow and processing modules. In particular, four gray scale radiographic images (402-408), each generated using a different energy level, are provided to an effective Z-value determination module 410. The effective Z-value determination module determines a pixel-level Z-value gray scale image 412.
The pixel-level Z-value gray scale image 412 can be provided to an image segmentation and layer analysis module 414. The segmentation and layer analysis module 414 segments the image and analyzes layers, as described above, to generate a layer corrected image representing true gray values, ROI coordinates, or both.
It will be appreciated that the modules, processes, systems, and sections described above can be implemented in hardware, software, or both. Also, the modules, processes systems, and sections can be implemented as a single processor or as a distributed processor. Further, it should be appreciated that the steps mentioned above may be performed on a single or distributed processor. Also, the processes, modules, and sub-modules described in the various figures of the embodiments above may be distributed across multiple computers or systems or may be co-located in a single processor or system. Exemplary structural embodiment alternatives suitable for implementing the modules, sections, systems, means, or processes described herein are provided below.
The modules, processors or systems described above can be implemented as a programmed general purpose computer, an electronic device programmed with microcode, a hard-wired analog logic circuit, software stored on a computer-readable medium or signal, a programmed kiosk, an optical computing device, a GUI on a display, a networked system of electronic and/or optical devices, a special purpose computing device, an integrated circuit device, a semiconductor chip, and a software module or object stored on a computer-readable medium or signal, for example.
Embodiments of the method and system (or their sub-components or modules), may be implemented on a general-purpose computer, a special-purpose computer, a programmed microprocessor or microcontroller and peripheral integrated circuit element, an ASIC or other integrated circuit, a digital signal processor, a hardwired electronic or logic circuit such as a discrete element circuit, a programmed logic circuit such as a PLD, PLA, FPGA, PAL, or the like. In general, any process capable of implementing the functions or steps described herein can be used to implement embodiments of the method, system, or a computer program product (software program).
Furthermore, embodiments of the disclosed method, system, and computer program product may be readily implemented, fully or partially, in software using, for example, object or object-oriented software development environments that provide portable source code that can be used on a variety of computer platforms. Alternatively, embodiments of the disclosed method, system, and computer program product can be implemented partially or fully in hardware using, for example, standard logic circuits or a VLSI design. Other hardware or software can be used to implement embodiments depending on the speed and/or efficiency requirements of the systems, the particular function, and/or particular software or hardware system, microprocessor, or microcomputer being utilized. Embodiments of the method, system, and computer program product can be implemented in hardware and/or software using any known or later developed systems or structures, devices and/or software by those of ordinary skill in the applicable art from the function description provided herein and with a general basic knowledge of the computer, image processing, radiographic, and/or threat detection arts.
Moreover, embodiments of the disclosed method, system, and computer program product can be implemented in software executed on a programmed general purpose computer, a special purpose computer, a microprocessor, or the like. Also, the method of this invention can be implemented as a program embedded on a personal computer such as a JAVA® or CGI script, as a resource residing on a server or image processing workstation, as a routine embedded in a dedicated processing system, or the like. The method and system can also be implemented by physically incorporating the method into a software and/or hardware system, such as the hardware and software systems of multi-energy radiographic cargo inspection systems.
It is, therefore, apparent that there is provided, in accordance with the present invention, a method, computer system, and computer software program for image segmentation. While this invention has been described in conjunction with a number of embodiments, it is evident that many alternatives, modifications and variations would be or are apparent to those of ordinary skill in the applicable arts. Accordingly, Applicant intends to embrace all such alternatives, modifications, equivalents and variations that are within the spirit and scope of this invention.

Claims

1. A system for segmenting radiographic images of a cargo container, the system comprising:

an object segmentation recognition module adapted to perform a series of functions including:

receiving a plurality of radiographic images of a cargo container, each image generated using a different energy level;

segmenting each of the radiographic images using one or more segmentation modules to generate segmentation data representing one or more image segments;

identifying image layers within the radiographic images using a plurality of layer analysis modules by providing the plurality of radiographic images and the segmentation data as input to the layer analysis modules, and determining adjusted atomic number values for an atomic number image based on the image layers;

adjusting the atomic number image based on the adjusted atomic number values for the regions of interest to generate an adjusted atomic number image;

identifying regions of interest within the adjusted atomic number image based on an image characteristic; and

providing coordinates of each region of interest and the adjusted atomic number image as output.

2. The system of claim 1, wherein the plurality of radiographic images are generated using four energy levels.

3. The system of claim 1, wherein identifying regions of interest includes comparing an estimated atomic value of each image segment to a threshold value.

4. The system of claim 1, wherein the plurality of layer analysis modules include a first layer analysis module and a second layer analysis module and the function of identifying image layers includes combining the output of the first layer analysis module with the second layer analysis module.

5. The system of claim 1, wherein the image characteristic is an estimated atomic value of a portion of the atomic number image.

6. The system of claim 1, wherein the image characteristic includes an image segment shape.

7. The system of claim 1, wherein the function of providing coordinates of each region of interest includes providing the coordinates of each region of interest and the corrected atomic number image to an operator station.

8. The system of claim 1, wherein the function of providing coordinates of each region of interest includes providing the coordinates of each region of interest and the corrected atomic number image to another system.

9. A method for segmenting radiographic images comprising:

providing a plurality of radiographic images;

providing a pixel level estimated atomic number image;

segmenting the radiographic images using one or more segmentation modules to generate segmentation data;

identifying image layers within each of the radiographic images;

analyzing the image layers using the plurality of radiographic images and the segmentation data in order to determine corrected atomic number values for objects in the images;

correcting the pixel level atomic number image based on the corrected atomic number values to generate an object level atomic number image;

identifying regions of interest within the object level atomic number image based on an image characteristic; and

outputting coordinates of each region of interest and the object level atomic number image as output.

10. The method of claim 9, wherein the plurality of radiographic images are images of a cargo container.

11. The method of claim 9, wherein the plurality of radiographic images includes four images each image being generated using a different energy level.

12. The method of claim 9, wherein identifying regions of interest includes comparing an estimated atomic value of each image object to a threshold value.

13. The method of claim 9, wherein the step of identifying image layers includes using a first layer analysis module and a second layer analysis module and combining the output of the first layer analysis module with the second layer analysis module.

14. The method of claim 9, wherein the image characteristic is an estimated atomic value of a portion of the object level atomic number image.

15. A radiographic image segmentation apparatus comprising:

means for receiving a radiographic image;

means for segmenting each of the radiographic images using one or more segmentation modules to generate segmentation data;

means for identifying image layers to produce image layer data;

means for identifying regions of interest based on the segmentation data and image layer data;

means for analyzing the image layers in order to determine corrected atomic number values for the regions of interest;

means for correcting an atomic number image based on the corrected atomic number values for the regions of interest; and

means for outputting region of interest coordinates and the corrected atomic number image as output.

16. The apparatus of claim 15, wherein the radiographic image is an image of a cargo container.

17. The apparatus of claim 15, wherein the means for receiving a radiographic image further includes means for receiving a plurality of radiographic images.

18. The apparatus of claim 17, wherein each radiographic image is generated using a different energy level.

19. The apparatus of claim 18, wherein the means for analyzing the image layers includes using the plurality of radiographic images and the segmentation data.

20. The apparatus of claim 15, wherein the means for identifying image layers includes first means for identifying image layers and second means for identifying image layers, the image layer data being produced using output from the first means for identifying image layers and output from the second means for identifying image layers.