WO2016107009A1 - 检查货物的方法和系统 - Google Patents

检查货物的方法和系统 Download PDF

Info

Publication number
WO2016107009A1
WO2016107009A1 PCT/CN2015/076421 CN2015076421W WO2016107009A1 WO 2016107009 A1 WO2016107009 A1 WO 2016107009A1 CN 2015076421 W CN2015076421 W CN 2015076421W WO 2016107009 A1 WO2016107009 A1 WO 2016107009A1
Authority
WO
WIPO (PCT)
Prior art keywords
goods
cargo
information
type
region
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2015/076421
Other languages
English (en)
French (fr)
Inventor
康克军
陈志强
张丽
赵自然
刘耀红
张健
顾建平
李强
张多坤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tsinghua University
Nuctech Co Ltd
Original Assignee
Tsinghua University
Nuctech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tsinghua University, Nuctech Co Ltd filed Critical Tsinghua University
Priority to MYPI2016703398A priority Critical patent/MY184775A/en
Priority to BR112016021802-7A priority patent/BR112016021802B1/pt
Priority to KR1020167026867A priority patent/KR101811270B1/ko
Priority to SG11201607652UA priority patent/SG11201607652UA/en
Priority to JP2017501456A priority patent/JP6445127B2/ja
Publication of WO2016107009A1 publication Critical patent/WO2016107009A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01V—GEOPHYSICS; GRAVITATIONAL MEASUREMENTS; DETECTING MASSES OR OBJECTS; TAGS
    • G01V5/00—Prospecting or detecting by the use of ionising radiation, e.g. of natural or induced radioactivity
    • G01V5/20—Detecting prohibited goods, e.g. weapons, explosives, hazardous substances, contraband or smuggled objects
    • G01V5/22—Active interrogation, i.e. by irradiating objects or goods using external radiation sources, e.g. using gamma rays or cosmic rays
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N23/00—Investigating or analysing materials by the use of wave or particle radiation, e.g. X-rays or neutrons, not covered by groups G01N3/00 – G01N17/00, G01N21/00 or G01N22/00
    • G01N23/02—Investigating or analysing materials by the use of wave or particle radiation, e.g. X-rays or neutrons, not covered by groups G01N3/00 – G01N17/00, G01N21/00 or G01N22/00 by transmitting the radiation through the material
    • G01N23/04—Investigating or analysing materials by the use of wave or particle radiation, e.g. X-rays or neutrons, not covered by groups G01N3/00 – G01N17/00, G01N21/00 or G01N22/00 by transmitting the radiation through the material and forming images of the material
    • G01N23/043—Investigating or analysing materials by the use of wave or particle radiation, e.g. X-rays or neutrons, not covered by groups G01N3/00 – G01N17/00, G01N21/00 or G01N22/00 by transmitting the radiation through the material and forming images of the material using fluoroscopic examination, with visual observation or video transmission of fluoroscopic images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection

Definitions

  • Embodiments of the present invention relate to inspection of goods, and in particular to inspection of goods based on radiation images of the cargo.
  • a method of inspecting goods comprising the steps of: obtaining a transmission image of the inspected goods; processing the transmission image to obtain a region of interest; performing feature extraction on the region of interest, And determining, according to the extracted features, the cargo information of the inspected cargo; and providing a recommended processing opinion on the cargo based on the determined cargo information and at least part of the information in the customs declaration.
  • the cargo information is specifically a stacking mode of the cargo
  • the step of performing feature extraction on the region of interest includes: extracting, from the region of interest, description information of a stacking mode of the inspected cargo;
  • the description information determines the probability of a stacking mode of goods in the region of interest of the transmitted image.
  • the step of extracting description information of the stacking mode of the inspected goods for the region of interest comprises: partitioning the region of interest, and performing edge extraction on the region image; A texture feature is extracted from the effective position in the extracted edge; the nearest neighbor element of the texture feature is found from the dictionary to obtain a histogram description feature of the region, wherein the dictionary is established based on the training image sample.
  • the dictionary is established as follows: equally spaced sampling is performed in each sample image, and texture features are extracted at each position; texture features extracted at all positions are clustered to obtain feature descriptions of multiple cluster centers, Form a dictionary.
  • the cargo area is extracted for the inspected goods and the no-cargo area is rejected by threshold definition to determine the area of interest.
  • the step of giving a recommendation processing opinion on the goods based on the determined cargo information and at least part of the information in the customs declaration comprises:
  • the consistency probability is determined by multiplying the cargo type probability of the stacking mode determined based on the customs declaration by the probability of the stacking mode determined based on the transmission image, and if the consistency probability is lower than the predetermined value, determining the goods and customs The goods in the order do not match.
  • the cargo information is specifically a cargo type
  • the step of performing feature extraction on the region of interest includes: extracting, from the region of interest, description information of a cargo type of the inspected cargo; wherein, based on the description Information determining the probability of the cargo type of the cargo in the region of interest of the transmission image; using the probability and combining the import and export company, the import and export time, and the importing and exporting country to obtain a probability value of the cargo type and the cargo type on the customs declaration .
  • the probability of the type of cargo is determined based on atomic number and electron density.
  • the step of giving a recommended treatment opinion for the goods based on the determined cargo information and at least part of the information in the customs declaration includes: comparing the predicted type of goods with the type of the goods of the customs declaration, and the export control The degree of analysis, to establish a cost model, and then give recommendations for the treatment of the goods.
  • the method further comprises the steps of: using an image content retrieval engine to obtain a list of goods similar to the goods to be inspected; and establishing a distribution relationship according to an import and export company, an import and export country, and an import and export time associated with the list of goods; Determining whether the goods are normal goods based on the established distribution relationship description.
  • the retrieved list of similar goods images is classified, and the probability of importing and exporting the same type of goods in different countries and different time periods is calculated.
  • the situation of importing and exporting similar goods in different time periods is displayed by establishing a time axis.
  • the method further comprises the step of prompting the user for a cargo area that deviates from the main stacking mode, a smaller cargo area that cannot be predicted by the cargo type, or a partial area that is significantly different from the global texture.
  • the method further comprises the step of: determining, by means of an interactive inference method, a cargo type of a single pile but a small pile of goods that cannot participate in the global information inference.
  • the method further comprises the step of inferring a type distribution, an average atomic number, or a position in the container of the goods that can present the texture based on the texture information of the local area.
  • the method further comprises the steps of: extracting a texture feature in the specified area; obtaining a texture block list having a similarity higher than a predetermined value based on the image content retrieval; determining the cargo type information of the texture block list, thereby establishing the presentation of the such Textured cargo type distribution.
  • the average atomic number of the inspected goods is obtained by a dual energy material identification algorithm.
  • the actual position of the designated area in the container is obtained using the image scaling scale and the pixel location of the designated area.
  • a system for inspecting goods comprising: a radiation imaging system that scans the inspected goods to obtain a transmission image of the inspected goods; and a data processing device that processes the transmitted images Obtaining a region of interest, performing feature extraction on the region of interest, determining cargo information of the inspected cargo according to the extracted feature, and giving the cargo based on the determined cargo information and at least part of the information in the customs declaration Recommended handling comments.
  • FIG. 1A and 1B are views showing the structure of a cargo inspection system according to an embodiment of the present invention.
  • FIG. 2 is a schematic flow chart describing a cargo inspection method according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram depicting four stacking modes involved in a scheme in accordance with an embodiment of the present invention.
  • FIG. 4 is a schematic diagram depicting layered inference of a mixture according to a scheme of an embodiment of the present invention.
  • FIG. 5 is a schematic diagram illustrating visualization of abnormal event inference in a scheme according to an embodiment of the present invention.
  • FIG. 6 is a flow chart describing a stacking mode prediction and recommendation processing opinion according to an embodiment of the present invention.
  • FIG. 7 is a flow chart describing a cargo type prediction, a risk level estimation, and a recommendation processing opinion according to an embodiment of the present invention
  • Figure 8 is a flow chart depicting similar cargo distribution inference in accordance with an embodiment of the present invention.
  • the intelligent analysis and inference system based on cargo X-ray image and customs declaration data is an automatic analysis of the cargo type, automatically giving easy to clamp the area, automatically inferring the composition of the mixed goods, automatically determining the abnormal cargo behavior, and giving according to the degree of risk
  • An intelligent analysis inference scheme that recommends processing opinions. For example, it can effectively solve the thorny problems that customs are facing, such as “Is the transported goods consistent with the type of customs declaration?”, “How to quickly guide the security personnel to locate the locations that are prone to trapping?”, “What may be mixed goods? Composition?”, "Is this import and export behavior abnormal?".
  • the system will grade different inference conclusions and corresponding processing opinions.
  • the present invention proposes that after obtaining the transmission image of the inspected goods, the transmission image is processed to obtain a region of interest, and then the region of interest is extracted, and the extracted feature is determined according to the extracted feature. Checking the cargo information of the cargo, and then giving recommendations for the processing of the cargo based on the determined cargo information and at least part of the information in the customs declaration.
  • FIGS. 1A and 1B are schematic structural views of an inspection system according to an embodiment of the present invention.
  • FIG. 1A shows a top plan view of the inspection system
  • FIG. 1B shows a front view of the inspection system.
  • the radiation source 110 generates X-rays that are collimated by the collimator 120 to perform a security check on the moving container truck 140
  • the detector 150 receives radiation that penetrates the truck, such as a computer.
  • the data processing device 160 of the class obtains a transmission image.
  • the transmission image of the container truck 140 is obtained by scanning
  • the transmission image is processed by the data processing device 160 to obtain a region of interest, such as a body cargo region, and then feature extraction is performed on the region of interest, and Determining the cargo information of the inspected goods according to the extracted features, and then giving a recommendation processing opinion on the goods based on the determined cargo information and at least part of the information in the customs declaration.
  • a transmission image of the inspected goods is obtained by the transmission scanning system.
  • the transmission scanning system For example, when the detected container truck passes the detection area at a certain speed, a picture can be produced. Perspective image.
  • the energy/dose, detector size, mechanical structure, vehicle speed, and beaming frequency of the source of the different scanning devices are different, so that the respective rates of image acquisition are slightly different.
  • the fluoroscopic image is first normalized as needed, ie, brightness normalization and resolution are uniformly scaled to, for example, 5*5 (mm/pixel).
  • step S22 the transmission image is processed to obtain a region of interest.
  • the pre-processing process can obtain the main cargo area. For example, the container area in the transmission image is extracted first, and then the main cargo area is extracted, for example by excluding the peripheral area of the main body cargo, to avoid interference and reduce the amount of invalid calculation.
  • the effects of air and wear-through areas in the image may also be removed.
  • the air region and the opaque region in the fluoroscopic image are directly detected by using a threshold method. When the goods are inspected, the influence of the air area and the impervious area can be avoided, which not only speeds up the detection but also reduces false alarms.
  • the cargo information herein may be a stacking mode of the cargo (eg, expressed as a probability value) or a type of cargo (eg, a probability value belonging to a particular cargo), and the like.
  • the type of cargo may be determined by atomic number and/or electron density and/or mass thickness, and the like.
  • a recommendation processing opinion for the goods is given based on the determined cargo information and at least part of the information in the customs declaration. For example, in combination with the cargo information obtained based on the image processing and the cargo information obtained from the customs declaration, the recommendation is given, and if the two are inconsistent, the passage is not allowed, or the classification is performed according to the consistency probability.
  • the cargo inspection system may include (1) global information inference, (2) local information inference, and (3) active learning.
  • the global information inference system is mainly divided into cargo stacking mode inference, cargo type inference, mixed cargo stratification inference, and abnormal event inference.
  • the cargo stacking mode is inferred by analyzing the cargo X-ray image to predict the cargo stacking mode, thereby inferring its probability of conforming to the customs cargo type (from the customs declaration data), and giving recommendations for processing.
  • the type of goods is specified by the customs HSCODE code.
  • different types of goods often correspond to different stacking modes, such as pulp goods.
  • the stacking mode 1 is presented, the grain goods generally exhibit the stacking mode 2, the alcoholic goods generally exhibit the stacking mode 3, and the motorcycle goods generally exhibit the stacking mode 4.
  • the use of image analysis techniques can give a prediction of the current cargo stacking pattern.
  • the type of customs declaration of the current goods can be given, and then the conventional stacking mode of the goods of this type can be obtained.
  • Combining the two aspects of information it can be inferred that the current goods are consistent with the type of customs goods (from the customs declaration data).
  • the recommended treatment opinions are given by the degree of consistency of the cargo stacking mode.
  • Figure 6 shows the flow of cargo stacking mode prediction and recommendation processing recommendations.
  • the scanned image is scanned and the container image is extracted from the scanned image of the container truck, and then the cargo area is extracted by the segmentation algorithm.
  • the segmentation algorithm uses an SRM (Statistical Region Merge) algorithm here.
  • SRM Statistical Region Merge
  • the segmented area is further processed, and the threshold-free method is used to eliminate the non-cargo area, and the area where the area does not exceed 1/3 of the entire container area is removed, and the remaining area is considered to be the main cargo area.
  • image feature extraction is performed on the body cargo area.
  • the Texton+BOW (Bag of Words) method is used here, but is not limited to this combination.
  • the description information of the stacking mode of the inspected goods is extracted for the region of interest, and the probability of the stacking mode of the goods in the region of interest of the transmission image is determined based on the description information.
  • the step of extracting the description information of the stacking mode of the inspected goods for the region of interest comprises: partitioning the region of interest and performing edge extraction on the region image; extracting the texture feature from the effective position in the extracted edge; The nearest neighboring element of the texture feature is found in the dictionary, resulting in a histogram description feature of the region, wherein the dictionary is established based on the training image samples.
  • the dictionary is established as follows: equally spaced sampling is performed in each sample image, and texture features are extracted at each position; texture features extracted at all positions are clustered to obtain feature descriptions of multiple cluster centers, Form a dictionary.
  • step S65 a SVM (Support Vector Machine) classification model is finally introduced for classification, and a prediction stacking mode is obtained.
  • SVM Small Vector Machine
  • the customs declaration type of the goods can be obtained, and the normal stacking mode of the goods can be obtained. For example, by comparing the cargo type probability of the stacking mode determined based on the customs declaration with the stacking mode determined based on the transmission image The probability is multiplied to determine the probability of consistency, and if the probability of consistency is below a predetermined value, the goods are judged to be inconsistent with the goods in the customs declaration. Use the following probability formula to calculate the cargo type consistency probability:
  • p(Cargo Stck Mode manifest ) indicates the probability of obtaining this type of cargo on the condition of the cargo stacking mode from the customs declaration
  • Stack Mode predict ) indicates that the cargo stacking mode predicted based on the transmission image is used as a condition , get the probability of the cargo stacking mode in the customs declaration.
  • the recommended treatment suggestions can be divided into four levels, which can be passed, seemingly different, and seem to require careful inspection and need to be carefully checked.
  • the type of goods is inferred by analyzing the X-ray image content of the goods, the dual-energy material identification information and the data on the customs declaration to predict the possible types of goods, thereby inferring the probability that they are consistent with the type of customs goods (from the customs declaration data), and Give the degree of risk and recommendations for treatment.
  • the type of goods is specified by the customs HSCODE code.
  • the prediction of the type of goods requires modeling of image content information, dual-energy material identification information and data from customs declarations, such as import and export companies, import and export countries, import and export times, etc., thus completing joint inference.
  • the probability of conforming to the type of goods on the customs declaration is finally given.
  • the cost model is established to give the risk level, and the recommended treatment plan is given based on this.
  • FIG. 7 is a flow chart describing a cargo type prediction, a risk level estimation, and a recommendation processing opinion according to an embodiment of the present invention.
  • container area extraction is performed on the scanned image of the container truck, and then the cargo area is extracted by the segmentation algorithm.
  • the segmentation algorithm herein employs an SRM algorithm.
  • the segmented area is further processed, and the threshold-free method is used to eliminate the non-cargo area, and the area where the area does not exceed 1/3 of the entire container area is removed, and the remaining area is considered to be the main cargo area.
  • step S74 image feature extraction is performed on the main cargo area.
  • the Texton+BOW (Bag of Words) method is used here, but is not limited to this combination.
  • step S75 an SVM (Support Vector Machine) classification model is introduced for classification, and a preliminary predicted cargo type is obtained.
  • step S76 the probabilistic model is used to combine customs declaration information, including import and export companies, import and export times, import and export countries.
  • the probability formula is as follows:
  • Imafe) represents the probability of predicting the type of goods based on the image content
  • Cargo) represents the probability of the country importing and exporting such goods
  • Cargo) indicates The probability of a company that exports such goods
  • Cargo) indicates the probability of the period of import and export of such goods.
  • the cost value is calculated using the following cost model based on tax and cargo control, and then at step S78, the degree of risk is determined based on the value of the generation:
  • T m is the tax rate applicable to the type of goods on the customs declaration
  • T p is the tax rate for the type of goods forecasted
  • C p is the forecasted type of goods is controlled goods, if it is 1, otherwise 0
  • ⁇ , ⁇ are parameters used to control the extent to which tax rates and controls affect the cost model.
  • the recommendations for giving recommendations can be divided into four levels, which can be passed, seemingly different, and seem to require careful examination and need to be carefully checked.
  • the stratified inference of mixed goods is to infer the composition of the mixed goods through a physical mathematical model.
  • a schematic diagram of hierarchical inference is shown in Figure 4.
  • a layer is constructed every 10 atomic numbers for material display in this segment.
  • An X-ray having an energy spectrum distribution is equivalently regarded as a single-energy X-ray of a certain energy.
  • the relationship between the input ray intensity and the output ray intensity is derived by establishing a decay law for the single energy X-rays passing through the mixture.
  • the method of supervised learning is used to solve the mass attenuation coefficient of different atomic number materials under unknown equivalent energy X-rays.
  • the sparse solution of the mass thickness can be calculated by the sparse method to infer the composition of the mixture.
  • I E ' is the X-ray dose at energy E'; I is the total dose after the X-ray passes through the substance; ⁇ (E', Z) is the mass attenuation coefficient, which is related to the atomic number and X-ray energy of the substance ;t m is the mass thickness, which is related to the atomic number of the substance.
  • I 0 represents the output dose of X-rays
  • I represents the dose after X passes through the substance
  • ⁇ eff (Z) is the equivalent mass attenuation coefficient
  • t m (Z) represents the mass thickness
  • a supervised sparse dictionary learning algorithm is employed, but is not limited thereto.
  • the specific algorithm is as follows:
  • the layered display of the mixture can be achieved based on the mixture composition inference algorithm described above.
  • Abnormal event inference is an anomaly event inference scheme based on cargo image content information, import and export companies, import and export countries, and import and export time information. Under normal circumstances, different import and export companies operate different import and export goods, different countries prefer to import and export specific types of goods, similar types of goods are often concentrated in a certain period of time.
  • the process of inference is to obtain a list of goods similar to the current scanned goods through the image content retrieval engine, and establish a distribution relationship diagram according to the corresponding import and export company, import and export country, import and export time (see Figure 5). Relying on the established distribution map and customs declaration data can intuitively indicate whether the current import and export of such goods is a normal behavior.
  • FIG. 8 is a flow chart depicting similar cargo distribution inference in accordance with an embodiment of the present invention.
  • the front end processing of the cargo image is substantially identical to the processing in the cargo stacking mode inference, so steps S81, S82 and S84 are the same as S61, S62 and S64 described above with reference to Fig. 6, and will not be described in detail herein.
  • step S85 after obtaining the feature description of the cargo image, the image content-based retrieval module is entered to obtain a cargo image list having a similarity of 80% or more.
  • the images in the list extract import and export countries, import and export companies, import and export time data from their corresponding customs declaration data.
  • the list of similar cargo images retrieved is classified, and the probability of importing and exporting the same type of goods in different countries and different time periods is calculated.
  • the visualization scheme of FIG. 5 is employed. By establishing a timeline, it shows the situation of similar imports and exports in different time periods.
  • the local information inference system is mainly divided into key area guidance, single pile cargo inference and user specified local area information inference.
  • the focus area guidance is primarily responsible for quickly guiding the user to focus on specific areas of the cargo X-ray image.
  • the specific areas mainly include (1) cargo areas that deviate from the main stacking mode, (2) smaller cargo areas that cannot predict the type of goods, and (3) local areas that are significantly different from the global texture. Often in these specific areas, prohibited items may be entrained.
  • (1), (2) Extraction of specific regions can be indirectly provided by image segmentation and region recognition. For areas where the stacking mode or cargo type cannot be identified, and the area is within the specified range, it can be considered as a specific area.
  • (3) Extraction of specific regions mainly based on supervised learning method to establish a texture similarity calculation model, and specific region detection based on the significance detection method.
  • the key areas are divided into three different types, areas that deviate from the main stacking mode, smaller areas that cannot be predicted by the type of goods, and local areas that are significantly different from the global texture.
  • the first two key areas can be indirectly given by cargo stacking model inference and cargo type inference.
  • To calculate the third focus area you need to give a definition of texture similarity.
  • the definition space of the human being is established by supervising the learning, thereby establishing the definition of the similarity of the texture block, but is not limited thereto.
  • the first step is to collect different texture blocks from the training cargo image.
  • the second step is to compare the similarity of texture blocks according to the subjective judgment of human beings.
  • the third step is to perform feature extraction on the texture block.
  • the Stochastic triplet embedding algorithm is used to establish a texture similarity comparison model, that is, the construction completes the understanding space.
  • the general method of saliency detection can be used to complete the extraction of the third key area.
  • the single pile of goods is inferred to be responsible for handling separate piles but the pile is small and cannot participate in the global letter. Infer the goods in the interest and infer the corresponding type of goods.
  • the main use of interactive inference methods which need to combine the image content information and user interaction information for joint inference. Joint inference is achieved by constructing a joint probability model of the cargo type and user interaction response under given image information.
  • the single pile of goods is inferred using an interactive inference method that relies on image content and human interaction to infer the identification of a single pile of goods.
  • the probability model is constructed:
  • z represents the cargo category
  • U represents the user's interactive response
  • x represents the image content.
  • x) can be obtained by a general image recognition algorithm, and p(U
  • the interaction information selected here is: selecting the most representative texture block of the single stack of goods, and selecting the goods similar to the image of the single stack from the candidate list.
  • the user specifies local area information inference, which is mainly responsible for inferring the distribution of the type of goods that can present such texture, the average atomic number, and the position in the container based on the texture information of the local area. It is inferred that the distribution of goods type mainly uses texture-based image retrieval technology. It is inferred that the average atomic number is mainly through the dual energy X-ray material recognition algorithm. Inferring the location in the container needs to be determined by the location of the area in the image and the scaling of the image.
  • the implementation steps are as follows:
  • the texture features of the specified area are extracted, preferably using a Fisher Vector.
  • a texture block list with a similarity of 80% or more is obtained.
  • the third part obtains the corresponding cargo type information through the retrieved texture block list, thereby establishing a cargo type distribution map that can present such texture.
  • the average atomic number is obtained by the dual energy material recognition algorithm.
  • the scaling scale of the image and the pixel location of the designated area get its coordinate position in the container.
  • the active learning system is responsible for actively discovering image samples that are difficult to discriminate based on the current model, and provides them with the help of annotations, and finally updates the model online.
  • the active learning module provides real-time services for global system inference and local system inference.
  • the active learning system is the underlying auxiliary system, which is mainly responsible for cargo stacking model speculation and cargo type speculation model services. Proactively discover samples that are difficult to identify based on the current model and update the model online.
  • aspects of the embodiments disclosed herein may be implemented in an integrated circuit as a whole or in part, as one or more of one or more computers running on one or more computers.
  • a computer program eg, implemented as one or more programs running on one or more computer systems
  • implemented as one or more programs running on one or more processors eg, implemented as one or One or more programs running on a plurality of microprocessors, implemented as firmware, or substantially in any combination of the above, and those skilled in the art, in accordance with the present disclosure, will be provided with design circuitry and/or write software and / or firmware code capabilities.
  • signal bearing media include, but are not limited to, recordable media such as floppy disks, hard drives, compact disks (CDs), digital versatile disks (DVDs), digital tapes, computer memories, and the like; and transmission-type media such as digital and / or analog communication media (eg, fiber optic cable, waveguide, wired communication link, wireless communication link, etc.).

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Geophysics (AREA)
  • General Life Sciences & Earth Sciences (AREA)
  • High Energy & Nuclear Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Quality & Reliability (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Pathology (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • Immunology (AREA)
  • Chemical & Material Sciences (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Multimedia (AREA)
  • Analysing Materials By The Use Of Radiation (AREA)
  • Image Analysis (AREA)
  • Alarm Systems (AREA)

Abstract

一种检查货物的方法和系统。该方法包括步骤：获得被检查货物的透射图像（S21）；对所述透射图像进行处理，得到感兴趣区域（S22）；对所述感兴趣区域进行特征提取，并根据提取的特征确定所述被检查货物的货物信息（S23）；基于所确定的货物信息和报关单中的至少部分信息给出对该货物的推荐处理意见（S24）。根据上述方法，能够协助判图员准确判断所涉及的货物是否允许通过。

Description

检查货物的方法和系统 技术领域
本发明的实施例涉及货物的检查，具体涉及基于货物的辐射图像来对货物进行检查。
背景技术
在目前辐射安检领域，对X射线图像的分析仅仅停留在双能材料识别，丝分辨，穿透测试等方面。这些方面往往提供给安检人员对货物最底层的理解，如货物材料组成、形状。却无法回答像这样的问题，“这个货物是否与其报关货物相符呢？”，“易于发生夹藏的区域在哪里呢”，“在这个时间段内，这家公司运输这个货物到那个国家是否异常呢？”，“放在旁边的那一小堆货物是什么？”。目前这些问题都需要凭借安检人员的丰富经验以及对图像内容的仔细推敲才能回答。随着海关吞吐量的增多，寻找一个智能化的解决方案是一个必然的方向。
发明内容
鉴于现有技术中的一个或多个问题，提出了一种对货物进行检查方法和系统，能够智能地进行推断并给出处理建议。
在本发明的一个方面，提出了一种检查货物的方法，包括步骤：获得被检查货物的透射图像；对所述透射图像进行处理，得到感兴趣区域；对所述感兴趣区域进行特征提取，并根据提取的特征确定所述被检查货物的货物信息；基于所确定的货物信息和报关单中的至少部分信息给出对该货物的推荐处理意见。
优选地，所述货物信息具体为货物的堆放模式，所述对所述感兴趣区域进行特征提取的步骤包括：对所述感兴趣区域提取被检查货物的堆放模式的描述信息；其中，基于所述描述信息确定所述透射图像的感兴趣区域中货物的堆放模式的概率。
优选地，对所述感兴趣区域提取被检查货物的堆放模式的描述信息的步骤包括：对感兴趣区域进行分区，并对区域图像进行边缘提取； 从提取的边缘中的有效位置处抽取纹理特征；从字典中找到所述纹理特征的最邻近的元素，得到所述区域的直方图描述特征，其中所述字典是基于训练图像样本而建立的。
优选地，如下建立所述字典：在每个样本图像中进行等间距采样，并在每个位置抽取纹理特征；对所有位置抽取的纹理特征进行聚类，得到多个聚类中心的特征描述，形成字典。
优选地，对所述被检查货物提取货物区域并且通过阈值限定来剔除无货物区域，从而确定感兴趣区域。
优选地，基于所确定的货物信息和报关单中的至少部分信息给出对该货物的推荐处理意见的步骤包括：
通过将基于报关单确定的堆放模式的货物类型概率与基于透射图像确定的堆放模式的概率相乘来确定一致性概率，并且在该一致性概率低于预定值的情况下，判断该货物与报关单中的货物不符。
优选地，所述货物信息具体为货物类型，所述对所述感兴趣区域进行特征提取的步骤包括：对所述感兴趣区域提取被检查货物的货物类型的描述信息；其中，基于所述描述信息确定所述透射图像的感兴趣区域中货物的货物类型的概率；使用所述概率并联合进出口公司、进出口时间、和进出口国家得到该货物与报关单上货物类型一致性的概率值。
优选地，对货物类型的概率是根据原子序数和电子密度而确定的。
优选地，基于所确定的货物信息和报关单中的至少部分信息给出对该货物的推荐处理意见的步骤包括：通过比较预测的货物类型和报关单货物类型的税费差异，和对出口管制程度的分析，来建立代价模型，进而给出对该货物的推荐处理意见。
优选地，所述的方法还包括步骤：利用图像内容检索引擎，获得与被检查货物相似的货物列表；根据与所述货物列表相关的进出口公司、进出口国家和进出口时间建立分布关系描述；基于所建立的分布关系描述确定所述货物是否属于正常货物。
优选地，根据货物的类型，对检索到的相似货物图像列表进行分类，并计算不同国家、不同公司在不同时间段内进出口相同类型货物的概率。
优选地，通过建立时间轴，来显示不同时间段内进出口相似货物的情况。
优选地，在每个时间段内，通过不同的图形显示不同进出口国家对，并显示不同进出口公司进出口相似货物的概率分布。
优选地，所述的方法还包括步骤：向用户提示偏离于主体堆放模式的货物区域、无法进行货物类型预测的较小货物区域、或与全局纹理有较大差异的局部区域。
优选地，所述的方法还包括步骤：采用交互式的推断方法确定单独成堆但堆较小无法参与到全局信息推断当中的货物的货物类型。
优选地，所述的方法还包括步骤：根据局部区域的纹理信息推断可以呈现这种纹理的货物的类型分布、平均原子序数、或在集装箱中的位置。
优选地，所述的方法还包括步骤：抽取指定区域内的纹理特征；基于图像内容检索，得到相似度高于预定值的纹理块列表；确定纹理块列表的货物种信息，从而建立呈现此类纹理的货物类型分布。
优选地，通过双能材料识别算法得到被检查货物的平均原子序数。
优选地，利用图像缩放尺度和指定区域的像素位置得到指定区域在集装箱中的实际位置。
在本发明的另一方面，提出了一种检查货物的系统，包括：辐射成像系统，对被检查货物进行扫描，以获得被检查货物的透射图像；数据处理装置，对所述透射图像进行处理，得到感兴趣区域，对所述感兴趣区域进行特征提取，根据提取的特征确定所述被检查货物的货物信息，基于所确定的货物信息和报关单中的至少部分信息给出对该货物的推荐处理意见。
利用上述的方案，通过对射线图像内容和报关单数据的分析，能够给出不同的推断结论和相应的处理意见，方便判图员提高效率和准确度。
附图说明
为了更好的理解本发明，将根据以下附图对本发明的实施例进行描述：
图1A和图1B示出了根据本发明实施例的货物检查系统的结构示意图；
图2是描述根据本发明实施例的货物检查方法的示意性流程图；
图3是描述根据本发明实施例的方案中涉及的四种堆放模式的示意图；
图4是描述根据本发明实施例的方案对混合物分层推断的示意图；
图5是描述根据本发明实施例的方案中异常事件推断的可视化示意图；
图6是描述根据本发明实施例的堆放模式预测及推荐处理意见的流程图；
图7是描述根据本发明实施例的货物类型预测、风险程度估计及推荐处理意见的流程图；
图8是描述根据本发明实施例的相似货物分布推断的流程图。
附图没有对实施例的所有电路或结构进行显示。贯穿所有附图相同的附图标记表示相同或相似的部件或特征。
具体实施方式
下面将详细描述本发明的具体实施例，应当注意，这里描述的实施例只用于举例说明，并不用于限制本发明。在以下描述中，为了提供对本发明的透彻理解，阐述了大量特定细节。然而，对于本领域普通技术人员显而易见的是：不必采用这些特定细节来实行本发明。在其他实例中，为了避免混淆本发明，未具体描述公知的电路、材料或方法。
在整个说明书中，对“一个实施例”、“实施例”、“一个示例”或“示例”的提及意味着：结合该实施例或示例描述的特定特征、结构或特性被包含在本发明至少一个实施例中。因此，在整个说明书的各个地方出现的短语“在一个实施例中”、“在实施例中”、“一个示例”或“示例”不一定都指同一实施例或示例。此外，可以以任何适当的组合和/或子组合将特定的特征、结构或特性组合在一个或多个实施例或示例中。 此外，本领域普通技术人员应当理解，在此提供的附图都是为了说明的目的，并且附图不一定是按比例绘制的。这里使用的术语“和/或”包括一个或多个相关列出的项目的任何和所有组合。
基于货物X射线图像和报关单数据的智能分析推断系统，是一个可以自动分析货物类型，自动给出易于夹藏区域，自动推断混合货物组成，自动判断异常运货行为，并根据风险程度给出推荐处理意见的智能化分析推断方案。例如，可以有效地解决海关正面临的棘手问题，如“运输的货物与其报关货物类型是否相符？”，“如何快速的引导安检人员定位易于发生夹藏的位置？”，“混合货物可能由什么组成？”，“这次进出口行为是否异常？”。通过对X射线图像内容和报关单数据的分析，系统将会分级给出不同的推断结论和相应的处理意见。
针对现有技术中的问题，本发明提出在获得被检查货物的透射图像后，对所述透射图像进行处理，得到感兴趣区域，然后对感兴趣区域进行特征提取，并根据提取的特征确定被检查货物的货物信息，进而基于所确定的货物信息和报关单中的至少部分信息给出对该货物的推荐处理意见。
图1A和图1B是根据本发明一个实施方式的检查系统的结构示意图。图1A示出了检查系统的俯视示意图，图1B示出了检查系统的正视示意图。如图1A和图1B所示，射线源110产生X射线，经过准直器120准直后，对移动的集装箱卡车140进行安全检查，由探测器150接收穿透卡车的射线，在诸如计算机之类的数据处理装置160得到透射图像。
根据本发明的实施例，在通过扫描得到集装箱卡车140的透射图像后，在数据处理装置160对透射图像进行处理，得到感兴趣区域，例如主体货物区域，然后对感兴趣区域进行特征提取，并根据提取的特征确定被检查货物的货物信息，进而基于所确定的货物信息和报关单中的至少部分信息给出对该货物的推荐处理意见。
图2是描述根据本发明实施例的检查方法的流程图。如图2所示，在步骤S21，通过透射扫描系统获得被检查货物的透射图像。例如，当被检测集装箱卡车以一定的速度经过检测区域时，就可以产生一幅 透视图像。不同扫描设备的射线源的能量/剂量、探测器尺寸、机械结构、车速和出束频率的不同，使得获取图像的分别率略有不同。在一些实施例中，根据需要对透视图像先做归一化处理，即亮度归一化和分辨率统一缩放到例如5*5(mm/pixel)。
在步骤S22,对所述透射图像进行处理，得到感兴趣区域。预处理的过程可以得到主体货物区域。例如，先提取透射图像中的集装箱区域，然后提取主体货物区，例如通过排除主体货物的外围区域，以避免干扰和减少无效的计算量。
此外，在一些实施例中，也可以去除图像中的空气区域和穿不透区域的影响。本发明实施例中直接采用阈值的方法检测出透视图像中的空气区域和穿不透区域。在进行货物检测的时候，可以避免空气区域和穿不透区域的影响，不仅可以加快检测速度，还可以减少误报。
在步骤S23，对所述感兴趣区域进行特征提取，并根据提取的特征确定所述被检查货物的货物信息。根据不同的实施例，这里的货物信息可以为货物的堆放模式(例如以概率值表示)或者货物的类型(例如属于某种特定货物的概率值)等等。在其他实施例中，货物的类型可以通过原子序数和/或电子密度和/或质量厚度等来确定。
在步骤S24，基于所确定的货物信息和报关单中的至少部分信息给出对该货物的推荐处理意见。例如结合基于图像处理得到的货物信息和从报关单得到的货物信息给出推荐意见，如二者之间不一致，则不允许通过，或者按照一致性概率分级处理。
在本发明实施例中，货物检查系统可以包括，(1)全局信息推断，(2)局部信息推断，(3)主动学习。全局信息推断系统主要分为货物堆放模式推断，货物类型推断，混合货物分层推断，异常事件推断。
货物堆放模式推断是通过分析货物X射线图像来预测货物堆放模式，从而推断其与报关货物类型(来自报关单数据)相一致的概率，并给出推荐处理意见。
货物类型由海关HSCODE编码指定。货物在集装箱中的堆放存在着一般规律，主要可以分为四种常见的堆放模式，如图3所示。一般情况下，不同的货物类型常常对应着不同的堆放模式，例如纸浆货物 一般呈现堆放模式1，谷类货物一般呈现堆放模式2，酒类货物一般呈现堆放模式3，摩托车货物一般呈现堆放模式4。使用图像分析技术可以给出对当前货物堆放模式的预测。通过检索报关单上的信息，可以给出当前货物的报关类型，进而得到此类型货物的常规堆放模式。结合两方面信息，可以推断当前货物与报关货物类型(来自报关单数据)相一致的概率。通过货物堆放模式的一致性程度，给出推荐处理意见。
图6示出了货物堆放模式预测及推荐处理建议流程。在步骤S61，S62和S63，扫描得到扫描图像并对集装箱卡车的扫描图像进行集装箱区域提取，然后通过分割算法提取货物区域。优选地，分割算法这里采用SRM(Statistical Region Merge)算法。对分割后的区域进一步处理，采用阈值限制方法剔除无货物区域，对面积没有超过整个集装箱面积1/3的区域剔除，剩下的区域认为是主体货物区域。
在步骤S64，对主体货物区域进行图像特征提取。优选地，这里采用Texton+BOW(Bag of Words，词袋)方法，但不限于这种组合。例如对所述感兴趣区域提取被检查货物的堆放模式的描述信息，基于描述信息确定所述透射图像的感兴趣区域中货物的堆放模式的概率。
优选地，对感兴趣区域提取被检查货物的堆放模式的描述信息的步骤包括：对感兴趣区域进行分区，并对区域图像进行边缘提取；从提取的边缘中的有效位置处抽取纹理特征；从字典中找到所述纹理特征的最邻近的元素，得到所述区域的直方图描述特征，其中所述字典是基于训练图像样本而建立的。优选地，如下建立所述字典：在每个样本图像中进行等间距采样，并在每个位置抽取纹理特征；对所有位置抽取的纹理特征进行聚类，得到多个聚类中心的特征描述，形成字典。
在步骤S65，最后引入SVM(Support Vector Machine)分类模型进行分类，并得到预测堆放模式。
在步骤S66，S67和S68，通过检索报关单，可以得到货物的报关类型，进而可以得到货物的常规堆放模式。例如，通过将基于报关单确定的堆放模式的货物类型概率与基于透射图像确定的堆放模式 的概率相乘来确定一致性概率，并且在该一致性概率低于预定值的情况下，判断该货物与报关单中的货物不符。使用如下概率公式计算货物类型一致性概率：
p(Cargo|Stack Modepredict)＝
p(Cargo|Stack Modemanifest)p(Stack Modemanifest|Stack Modepredict)
其中p(Cargo Stck Modemanifest)表示以来自报关单的货物堆放模式为条件，得到此类型货物的概率；p(Stack Modemanifest|Stack Modepredict)表示以基于透射图像预测得到的货物堆放模式为条件，得到报关单中的货物堆放模式的概率。
此外，推荐处理建议可以分为四个等级，可以通过，似乎有些不同，似乎需要仔细检查，需要仔细检查。
货物种类推断是通过分析货物X射线图像内容，双能材料识别信息以及报关单上的数据来预测货物可能的类型，从而来推断其与报关货物类型(来自报关单数据)相一致的概率，并给出风险程度和推荐处理意见。货物类型由海关HSCODE编码指定。对货物类型的预测需要对图像内容信息，双能材料识别信息和来自报关单上的数据，如进出口公司，进出口国家，进出口时间等信息进行建模，从而完成联合推断。通过推断结果，最终给出其与报关单上的货物类型相一致的概率。通过对预测货物类型和报关货物类型(来自报关单数据)的税费差异，出口管制程度的分析，来建立代价模型，从而给出风险程度，并以此为依据给出推荐处理方案。
图7是描述根据本发明实施例的货物类型预测、风险程度估计及推荐处理意见的流程图。在步骤S71，S72，和S73，对集装箱卡车的扫描图像进行集装箱区域提取，然后通过分割算法提取货物区域。优选地，分割算法这里采用SRM算法。对分割后的区域进一步处理，采用阈值限制方法剔除无货物区域，对面积没有超过整个集装箱面积1/3的区域剔除，剩下的区域认为是主体货物区域。
在步骤S74，对主体货物区域进行图像特征提取，优选地，这里采用Texton+BOW(Bag of Words)方法，但不限于这种组合。优选地，在步骤S75，引入SVM(Support Vector Machine)分类模型进行分类，并得到初步预测货物类型。在步骤S76，使用概率模型联合报关单信息，其中包括进出口公司，进出口时间，进出口国家。
概率公式如下：
Figure PCTCN2015076421-appb-000001
其中Z是归一化常数，p(Cargo|Imafe)表示根据图像内容预测得到货物类型的概率，p(Country|Cargo)表示进出口此类货物的国家的概率，p(Company|Cargo)表示进出口此类货物的公司的概率，p(Time|Cargo)表示进出口此类货物的时期的概率。
在步骤S77，利用下面基于税费和货物管制的代价模型来计算代价值，然后在步骤S78，根据代价值确定风险程度：
Figure PCTCN2015076421-appb-000002
其中Tm为针对于报关单上的货物类型需征收的税率；Tp是针对于预测货物类型需征收的税率；Cp是预测货物类型是否是管制货物，是则为1，否则为0；α,β为参数用来控制税率和管制对代价模型的影响程度。 在步骤S79，给出推荐处理建议可以分为四个等级，可以通过，似乎有些不同，似乎需要仔细检查，需要仔细检查。
混合货物分层推断是通过物理数学模型来推断混合货物的组成成分。分层推断示意图见图4。优选地，按每隔10个原子序数构建一个层，进行此段中的物质显示。将具有能谱分布的X射线等效地看成某一能量的单能X射线。从而整个物理过程简化成某个等效单能X射线穿过混合物质的过程。通过建立单能X射线穿过混合物质的衰减规律公式，导出输入射线强度和输出射线强度的关系。通过有监督学习的方法求解不同原子序数物质在未知的等效能量X射线下的质量衰减系数。在实际进行混合物质推断时，可以通过稀疏方法，来计算质量厚度的稀疏解，从而推断混合物的组成。
对于加速器产生的具有能谱分布X射线，其穿过物质的衰减规律如下：
Figure PCTCN2015076421-appb-000003
其中IE‘为在能量为E′的X射线剂量；I为X射线穿过物质后的总剂量；μ(E′,Z)为质量衰减系数，其与物质的原子序数和X射线能量有关；tm为质量厚度，其与物质的原子序数有关。
用μeff(Z)表示在指定能谱分布X射线下的质量衰减系数，其只与物质的原子序数有关，并将这一项定义为等效质量衰减系数。从而上述公式可以写成：
Figure PCTCN2015076421-appb-000004
其中I0表示X射线的输出剂量，I表示X穿过物质后的剂量，μeff(Z)为等效质量衰减系数，tm(Z)表示质量厚度。
对于混合物，则上述公式可以写成
Figure PCTCN2015076421-appb-000005
计算透明度ln T：
Figure PCTCN2015076421-appb-000006
对于双能系统存在高能和低能X射线能谱，因而存在
Figure PCTCN2015076421-appb-000007
简化成如下矩阵表达形式
Y＝MX
其中
Figure PCTCN2015076421-appb-000008
由于针对于不同能谱的等效质量衰减系数并不是特定常数，因而无法通过实验得出。优选地，在本发明中，采用有监督稀疏字典学习算法，但不限于此。具体算法如下：
1)字典学习，建立如下优化函数：
Figure PCTCN2015076421-appb-000009
2)基于训练样本对{X,Y}，进行字典学习，得到矩阵M。
3)混合物组成成分推断，建立如下优化函数：
Figure PCTCN2015076421-appb-000010
4)最小化上述函数得到稀疏解X，非零项即表示可能的组成成分。
基于上述描述的混合物成分推断算法便可以实现混合物分层显示。
异常事件推断，是一种基于货物图像内容信息，进出口公司，进出口国家，以及进出口时间信息的异常事件推断方案。一般情况下，不同的进出口公司经营不同的进出口货物，不同的国家偏好进出口特定货物类型，相似种类货物往往集中在一定时间内进出口。推断的过程是，通过图像内容检索引擎，获得与当前扫描货物相似的货物列表，并根据其对应的进出口公司，进出口国家，进出口时间建立分布关系图(见图5)。依靠所建立的分布图以及报关单数据可以直观地给出当前进出口此类货物是否属于常规行为。
图8是描述根据本发明实施例的相似货物分布推断的流程图。对货物图像的前端处理与在货物堆放模式推断中的处理方式基本一致，因此步骤S81，S82和S84与前面参考图6描述的S61，S62和S64相同，这里不再详细描述。
在步骤S85，在获得对货物图像的特征描述后，进入基于图像内容的检索模块，获得相似度达到80％以上的货物图像列表。根据列表中的图像，从其对应的报关单数据中提取进出口国家，进出口公司，进出口时间数据。根据货物的类型，对检索得到的相似货物图像列表进行分类，并计算不同国家，不同公司在不同的时间段内进出口同种类型货物的概率。优选地，在步骤S86，采用图5的可视化方案。通过建立时间轴，来显示不同时间段内进出口相似货物情况。在每个时间片段内，通过不同的椭圆图形显示不同进出口国家对，并在其内部 通过不同的椭圆形显示不同的进出口公司进出口相似货物的概率分布。依靠当前扫描货物的进出口国家，进出口公司，进出口时间以及报关货物类型，便可以直观地推断得到此类型货物，被当前公司进出口到指定国家是否属于常规事件。
局部信息推断系统主要分为重点区域引导，单堆货物推断和用户指定局部区域信息推断。
重点区域引导主要负责快速引导用户关注货物X射线图像中的特异性区域。特异性区域主要包括(1)偏离于主体堆放模式的货物区域，(2)无法进行货物类型预测的较小货物区域，(3)与全局纹理有较大差异的局部区域。往往在这些特异性区域中，可能夹带违禁物品。(1),(2)特异性区域的提取，均可通过图像分割和区域识别来间接提供。对无法识别堆放模式或货物类型的区域，并且其面积在指定范围内，均可认为是特异性区域。(3)特异性区域的提取，主要采用基于监督学习的方法建立纹理相似度计算模型，并根据显著性检测方法进行特异性区域检测。
重点区域分为三种不同类型，偏离主体堆放模式的区域、无法进行货物类型预测的较小区域、与全局纹理有较大差异的局部区域。第一二种重点区域均可以通过货物堆放模式推断和货物类型推断进行间接给出。计算第三种重点区域，需要给出纹理相似度的定义。优选地，通过监督学习的方式建立人的理解空间，从而建立纹理块的相似度的定义，但不限于此。
建立人的理解空间步骤：
第一步，从训练货物图像中采集不同的纹理块
第二步，根据人的主观判断，对纹理块的相似度进行比较
第三步，对纹理块进行特征抽取
第四步，采用Stochastic triplet embedding算法，建立纹理相似度比较模型，即构建完成理解空间。
纹理相似度模型建立完成之后，便可以采用显著性检测的一般方法完成第三种重点区域的抽取。
单堆货物推断主要负责处理单独成堆但堆较小无法参与到全局信 息推断当中的货物，并推断相应的货物类型。主要采用交互式的推断方法，其需要结合图像内容信息和用户交互信息进行联合推断。通过构建在给定图像信息下的货物类型和用户交互响应的联合概率模型来实现联合推断。
单堆货物推断采用交互式的推断方法，依靠图像内容以及人的交互来对单堆货物进行推断识别。优选地，构建概率模型：
p(z,U|x)＝p(U|z,x)p(z|x)
其中z表示货物类别，U表示用户的交互响应，x表示图像内容。p(z|x)可以通过一般的图像识别算法得到，p(U|z,x)需要通过人的交互响应得到。
交互信息的引入可以有效地增强单纯依靠图像内容所建立的模型的准确性。选择什么样的交互信息是至关重要的。它需要尽可能与可以直接通过算法从图像内容分析得到的信息成为互补，从而引入更多的有效信息帮助识别。
优选地，在这里选择的交互信息是：选择单堆货物最具代表性的纹理块、从候选列表中选择与此单堆货物图像相似的货物。
用户指定局部区域信息推断，主要负责根据局部区域的纹理信息来推断可以呈现这种纹理的货物类型分布，平均原子序数，以及在集装箱中的位置。推断货物类型分布主要采用的是基于纹理的图像检索技术。推断平均原子序数主要通过双能X射线材料识别算法。推断位于集装箱中的位置，需要通过此区域在图像中的位置以及图像放缩比例来确定。
例如，实现步骤如下：第一步，抽取指定区域的纹理特征，优选地，使用Fisher Vector。第二步，基于图像内容检索，得到相似度达到80％以上的纹理块列表。第三部，通过检索到的纹理块列表，得到其对应的货物种类信息，从而建立可以呈现此类纹理的货物类型分布图。第四步，通过双能材料识别算法，得到平均原子序数。第五步， 图像的放缩尺度和指定区域的像素位置得到其在集装箱中的坐标位置。
主动学习系统负责主动发现依靠当前模型难于判别的图像样本，并提供给用户使其帮助进行标注，最后进行模型的在线更新。主动学习模块为全局系统推断和局部系统推断进行实时服务。主动学习系统是底层辅助系统，主要负责货物堆放模型推测和货物类型推测模型服务。主动发现基于当前模型难于识别的样本，并在线更新模型。
以上的详细描述通过使用示意图、流程图和/或示例，已经阐述了检查方法和系统的众多实施例。在这种示意图、流程图和/或示例包含一个或多个功能和/或操作的情况下，本领域技术人员应理解，这种示意图、流程图或示例中的每一功能和/或操作可以通过各种结构、硬件、软件、固件或实质上它们的任意组合来单独和/或共同实现。在一个实施例中，本发明的实施例所述主题的若干部分可以通过专用集成电路(ASIC)、现场可编程门阵列(FPGA)、数字信号处理器(DSP)、或其他集成格式来实现。然而，本领域技术人员应认识到，这里所公开的实施例的一些方面在整体上或部分地可以等同地实现在集成电路中，实现为在一台或多台计算机上运行的一个或多个计算机程序(例如，实现为在一台或多台计算机系统上运行的一个或多个程序)，实现为在一个或多个处理器上运行的一个或多个程序(例如，实现为在一个或多个微处理器上运行的一个或多个程序)，实现为固件，或者实质上实现为上述方式的任意组合，并且本领域技术人员根据本公开，将具备设计电路和/或写入软件和/或固件代码的能力。此外，本领域技术人员将认识到，本公开所述主题的机制能够作为多种形式的程序产品进行分发，并且无论实际用来执行分发的信号承载介质的具体类型如何，本公开所述主题的示例性实施例均适用。信号承载介质的示例包括但不限于：可记录型介质，如软盘、硬盘驱动器、紧致盘(CD)、数字通用盘(DVD)、数字磁带、计算机存储器等；以及传输型介质，如数字和/或模拟通信介质(例如，光纤光缆、波导、有线通信链路、无线通信链路等)。
虽然已参照几个典型实施例描述了本发明，但应当理解，所用的术语是说明和示例性、而非限制性的术语。由于本发明能够以多种形 式具体实施而不脱离发明的精神或实质，所以应当理解，上述实施例不限于任何前述的细节，而应在随附权利要求所限定的精神和范围内广泛地解释，因此落入权利要求或其等效范围内的全部变化和改型都应为随附权利要求所涵盖。

Claims (20)

  1. 一种检查货物的方法，包括步骤：
    获得被检查货物的透射图像；
    对所述透射图像进行处理，得到感兴趣区域；
    对所述感兴趣区域进行特征提取，并根据提取的特征确定所述被检查货物的货物信息；
    基于所确定的货物信息和报关单中的至少部分信息给出对该货物的推荐处理意见。
  2. 如权利要求1所述的方法，其中所述货物信息具体为货物的堆放模式，所述对所述感兴趣区域进行特征提取的步骤包括：
    对所述感兴趣区域提取被检查货物的堆放模式的描述信息；
    其中，基于所述描述信息确定所述透射图像的感兴趣区域中货物的堆放模式的概率。
  3. 如权利要求2所述的方法，其中对所述感兴趣区域提取被检查货物的堆放模式的描述信息的步骤包括：
    对感兴趣区域进行分区，并对区域图像进行边缘提取；
    从提取的边缘中的有效位置处抽取纹理特征；
    从字典中找到所述纹理特征的最邻近的元素，得到所述区域的直方图描述特征，其中所述字典是基于训练图像样本而建立的。
  4. 如权利要求3所述的方法，其中如下建立所述字典：
    在每个样本图像中进行等间距采样，并在每个位置抽取纹理特征；
    对所有位置抽取的纹理特征进行聚类，得到多个聚类中心的特征描述，形成字典。
  5. 如权利要求1所述的方法，其中对所述被检查货物提取货物区域并且通过阈值限定来剔除无货物区域，从而确定感兴趣区域。
  6. 如权利要求2所述的方法，其中基于所确定的货物信息和报关单中的至少部分信息给出对该货物的推荐处理意见的步骤包括：
    通过将以报关单的堆放模式为条件得到的货物类型概率与以基于透射图像预测的堆放模式为条件得到报关单中堆放模式的概率相乘来确定一致性概率，并且在该一致性概率低于预定值的情况下，判 断该货物与报关单中的货物不符。
  7. 如权利要求1所述的方法，其中所述货物信息具体为货物类型，所述对所述感兴趣区域进行特征提取的步骤包括：
    对所述感兴趣区域提取被检查货物的货物类型的描述信息；
    其中，基于所述描述信息确定所述透射图像的感兴趣区域中货物的货物类型的概率；
    使用所述概率并联合进出口公司、进出口时间、和进出口国家得到该货物与报关单上货物类型一致性的概率值。
  8. 如权利要求7所述的方法，其中对货物类型的概率是根据原子序数和电子密度而确定的。
  9. 如权利要求7所述的方法，其中基于所确定的货物信息和报关单中的至少部分信息给出对该货物的推荐处理意见的步骤包括：通过比较预测的货物类型和报关单货物类型的税费差异，和对出口管制程度的分析，来建立代价模型，进而给出对该货物的推荐处理意见。
  10. 如权利要求1所述的方法，还包括步骤：
    利用图像内容检索引擎，获得与被检查货物相似的货物列表；
    根据与所述货物列表相关的进出口公司、进出口国家和进出口时间建立分布关系描述；
    基于所建立的分布关系描述确定所述货物是否属于正常货物。
  11. 如权利要求10所述的方法，其中根据货物的类型，对检索到的相似货物图像列表进行分类，并计算不同国家、不同公司在不同时间段内进出口相同类型货物的概率。
  12. 如权利要求11所述的方法，其中通过建立时间轴，来显示不同时间段内进出口相似货物的情况。
  13. 如权利要求10所述的方法，在每个时间段内，通过不同的图形显示不同进出口国家对，并显示不同进出口公司进出口相似货物的概率分布。
  14. 如权利要求1所述的方法，还包括步骤：
    向用户提示偏离于主体堆放模式的货物区域、无法进行货物类型预测的较小货物区域、或与全局纹理有较大差异的局部区域。
  15. 如权利要求1所述的方法，还包括步骤：
    采用交互式的推断方法确定单独成堆但堆较小无法参与到全局信息推断当中的货物的货物类型。
  16. 如权利要求1所述的方法，还包括步骤：
    根据局部区域的纹理信息推断可以呈现这种纹理的货物的类型分布、平均原子序数、或在集装箱中的位置。
  17. 如权利要求16所述的方法，还包括步骤：
    抽取指定区域内的纹理特征；
    基于图像内容检索，得到相似度高于预定值的纹理块列表；
    确定纹理块列表的货物种信息，从而建立呈现此类纹理的货物类型分布。
  18. 如权利要求17所述的方法，其中通过双能材料识别算法得到被检查货物的平均原子序数。
  19. 如权利要求17所述的方法，其中利用图像缩放尺度和指定区域的像素位置得到指定区域在集装箱中的实际位置。
  20. 一种检查货物的系统，包括：
    辐射成像系统，对被检查货物进行扫描，以获得被检查货物的透射图像；
    数据处理装置，对所述透射图像进行处理，得到感兴趣区域，对所述感兴趣区域进行特征提取，根据提取的特征确定所述被检查货物的货物信息，基于所确定的货物信息和报关单中的至少部分信息给出对该货物的推荐处理意见。
PCT/CN2015/076421 2014-12-30 2015-04-13 检查货物的方法和系统 Ceased WO2016107009A1 (zh)

Priority Applications (5)

Application Number Priority Date Filing Date Title
MYPI2016703398A MY184775A (en) 2014-12-30 2015-04-13 Systems and methods for inspecting cargoes
BR112016021802-7A BR112016021802B1 (pt) 2014-12-30 2015-04-13 Sistemas e métodos para inspecionar cargas
KR1020167026867A KR101811270B1 (ko) 2014-12-30 2015-04-13 화물을 검사하는 방법 및 그 시스템
SG11201607652UA SG11201607652UA (en) 2014-12-30 2015-04-13 Systems and methods for inspecting cargoes
JP2017501456A JP6445127B2 (ja) 2014-12-30 2015-04-13 貨物の検査方法およびそのシステム

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201410842364.4A CN105808555B (zh) 2014-12-30 2014-12-30 检查货物的方法和系统
CN201410842364.4 2014-12-30

Publications (1)

Publication Number Publication Date
WO2016107009A1 true WO2016107009A1 (zh) 2016-07-07

Family

ID=53039225

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/076421 Ceased WO2016107009A1 (zh) 2014-12-30 2015-04-13 检查货物的方法和系统

Country Status (9)

Country Link
US (1) US10074166B2 (zh)
EP (1) EP3040740B1 (zh)
JP (1) JP6445127B2 (zh)
KR (1) KR101811270B1 (zh)
CN (1) CN105808555B (zh)
BR (1) BR112016021802B1 (zh)
MY (1) MY184775A (zh)
SG (1) SG11201607652UA (zh)
WO (1) WO2016107009A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107621653A (zh) * 2017-07-17 2018-01-23 上海多克实业有限公司 一种快速检测集装箱是否为空的仪器和方法
JP2018146587A (ja) * 2017-03-08 2018-09-20 清華大学Ｔｓｉｎｇｈｕａ Ｕｎｉｖｅｒｓｉｔｙ 検査機器と車両の画像を分割する方法

Families Citing this family (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9111331B2 (en) * 2011-09-07 2015-08-18 Rapiscan Systems, Inc. X-ray inspection system that integrates manifest data with imaging/detection processing
EP3078944B1 (en) * 2015-04-07 2020-02-19 Mettler-Toledo, LLC Method of determining the mass of objects from a plurality of x-ray images taken at different energy level
CN106353828B (zh) * 2015-07-22 2018-09-21 清华大学 在安检系统中估算被检查物体重量的方法和装置
CN106706677B (zh) * 2015-11-18 2019-09-03 同方威视技术股份有限公司 检查货物的方法和系统
EP3505919B1 (en) * 2016-08-25 2024-07-17 Beijing Hualixing Technology Development Co., Ltd. Imaging device for use in vehicle security check and method therefor
WO2018060523A2 (en) 2016-09-30 2018-04-05 Ge Healthcare Bio-Sciences Corp Computer device for detecting an optimal candidate compound and methods thereof
CN108108744B (zh) * 2016-11-25 2021-03-02 同方威视技术股份有限公司 用于辐射图像辅助分析的方法及其系统
CN108734183A (zh) * 2017-04-14 2018-11-02 清华大学 检查方法和检查设备
CN109522913B (zh) * 2017-09-18 2022-07-19 同方威视技术股份有限公司 检查方法和检查设备以及计算机可读介质
US12398024B2 (en) * 2018-02-02 2025-08-26 Digital Logistics As Cargo detection and tracking
JP6863326B2 (ja) * 2018-03-29 2021-04-21 日本電気株式会社 選別支援装置、選別支援システム、選別支援方法及びプログラム
PL3797390T3 (pl) * 2018-05-21 2025-01-07 Smiths Detection-Watford Limited System i sposób do inspekcji przedmiotów
JP6754155B1 (ja) * 2018-10-01 2020-09-09 株式会社 システムスクエア 教師データ生成装置、検査装置およびコンピュータプログラム
CN113835130A (zh) * 2020-06-23 2021-12-24 同方威视技术股份有限公司 自动行走式的检查装置和自动分车方法
CN111915248A (zh) * 2020-07-16 2020-11-10 张文 社会车辆参与农产品运输的订单生成方法、装置、系统和介质
CN112232449B (zh) * 2020-12-14 2021-04-27 浙江大华技术股份有限公司 神经网络的训练方法、电子设备及存储介质
GB202109943D0 (en) * 2021-07-09 2021-08-25 Smiths Detection France S A S Image retrieval system
CN114004721B (zh) * 2021-11-01 2025-05-09 同方威视科技江苏有限公司 机检查验方法、装置与电子设备
CN116092096A (zh) * 2021-11-05 2023-05-09 同方威视技术股份有限公司 用于检验申报信息真实性的方法、系统、设备及介质
CN118014958A (zh) * 2024-02-05 2024-05-10 深圳云码通科技有限公司 一种包装箱缺陷检测方法及系统
CN117896506B (zh) * 2024-03-14 2024-07-05 广东电网有限责任公司 一种动态的现场作业场景可视化监控方法
CN120703848A (zh) * 2025-05-26 2025-09-26 电子科技大学 一种远距离脚底金属探测装置及方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080170655A1 (en) * 2007-01-17 2008-07-17 Ge Homeland Protection, Inc. Computed tomography cargo inspection system and method
CN103415787A (zh) * 2011-02-28 2013-11-27 得克萨斯A&M大学系统 货物检查系统
CN103917862A (zh) * 2011-09-07 2014-07-09 拉皮斯坎系统股份有限公司 整合舱单数据和成像/检测处理的x射线检查系统

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE3145227A1 (de) * 1981-11-13 1983-05-19 Heimann Gmbh, 6200 Wiesbaden Verfahren und vorrichtung zur untersuchung des inhaltes von containern
US6370222B1 (en) * 1999-02-17 2002-04-09 Ccvs, Llc Container contents verification
US7596275B1 (en) * 2004-03-01 2009-09-29 Science Applications International Corporation Methods and systems for imaging and classifying targets as empty or non-empty
WO2005087633A1 (ja) * 2004-03-12 2005-09-22 Mitsui Engineering & Shipbuilding Co., Ltd. コンテナ検査荷役方法およびコンテナ検査荷役システム
WO2006100674A2 (en) * 2005-03-21 2006-09-28 Yeda Research And Development Co. Ltd. Detecting irregularities
EP1886257A1 (en) * 2005-05-11 2008-02-13 Optosecurity Inc. Method and system for screening luggage items, cargo containers or persons
JP2007140729A (ja) 2005-11-16 2007-06-07 Ishikawajima Harima Heavy Ind Co Ltd 物品の位置及び姿勢を検出する方法および装置
CN101435783B (zh) * 2007-11-15 2011-01-26 同方威视技术股份有限公司 物质识别方法和设备

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080170655A1 (en) * 2007-01-17 2008-07-17 Ge Homeland Protection, Inc. Computed tomography cargo inspection system and method
CN103415787A (zh) * 2011-02-28 2013-11-27 得克萨斯A&M大学系统 货物检查系统
CN103917862A (zh) * 2011-09-07 2014-07-09 拉皮斯坎系统股份有限公司 整合舱单数据和成像/检测处理的x射线检查系统

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2018146587A (ja) * 2017-03-08 2018-09-20 清華大学Ｔｓｉｎｇｈｕａ Ｕｎｉｖｅｒｓｉｔｙ 検査機器と車両の画像を分割する方法
CN107621653A (zh) * 2017-07-17 2018-01-23 上海多克实业有限公司 一种快速检测集装箱是否为空的仪器和方法

Also Published As

Publication number Publication date
JP6445127B2 (ja) 2018-12-26
KR101811270B1 (ko) 2017-12-21
MY184775A (en) 2021-04-21
CN105808555A (zh) 2016-07-27
CN105808555B (zh) 2019-07-26
EP3040740B1 (en) 2019-06-26
BR112016021802A2 (pt) 2017-12-12
US20160189360A1 (en) 2016-06-30
KR20160130422A (ko) 2016-11-11
JP2017509903A (ja) 2017-04-06
US10074166B2 (en) 2018-09-11
EP3040740A1 (en) 2016-07-06
SG11201607652UA (en) 2016-11-29
BR112016021802B1 (pt) 2021-10-13

Similar Documents

Publication Publication Date Title
CN105808555B (zh) 检查货物的方法和系统
Zhang et al. Unified approach to pavement crack and sealed crack detection using preclassification based on transfer learning
US10303979B2 (en) System and method for classifying and segmenting microscopy images with deep multiple instance learning
KR102240058B1 (ko) 검사 방법과 검사 장비 및 컴퓨터 판독 가능한 매체
CN115295154B (zh) 肿瘤免疫治疗疗效预测方法、装置、电子设备及存储介质
US11804029B2 (en) Hierarchical constraint (HC)-based method and system for classifying fine-grained graptolite images
Khumancha et al. Lung cancer detection from computed tomography (CT) scans using convolutional neural network
CN114882215B (zh) 一种煤矸光电分选图像颗粒集料区域形选识别方法
CN109557114B (zh) 检查方法和检查设备以及计算机可读介质
Cengel et al. Automating egg damage detection for improved quality control in the food industry using deep learning
CN112949634A (zh) 一种铁路接触网鸟窝检测方法
Su et al. Utilizing pretrained convolutional neural networks for crack detection and geometric feature recognition in concrete surface images
Hou et al. Feature fusion for weld defect classification with small dataset
Liu et al. Multi-scale feature fusion for prediction of IDH1 mutations in glioma histopathological images
Çakmak et al. Deep learning for early diagnosis of lung cancer
Li et al. Artificial intelligence for histological subtype classification of breast cancer: combining multi‐scale feature maps and the recurrent attention model
Sethy et al. Maximizing steel slice defect detection: Integrating ResNet101 deep features with SVM via Bayesian optimization
Gao et al. Estimation of the convolutional neural network with attention mechanism and transfer learning on wood knot defect classification
Wang et al. Size‐adaptive mediastinal multilesion detection in chest CT images via deep learning and a benchmark dataset
Gurucharan et al. Advanced feature extraction techniques for machine learning based respiratory illnesses detection in chest radiographs
Durak et al. Coronal loop detection from solar images
Taş et al. Early Detection of Lung Metastases in Breast Cancer Using YOLOv10 and Transfer Learning: A Diagnostic Accuracy Study
Zaheer et al. Efficient and Interpretable Image Processing Approach for Crack Segmentation
Sarker et al. A means of assessing deep learning‐based detection of ICOS protein expression in colon cancer. Cancers. 202113: 3825
Høibø et al. Immunohistochemistry guided segmentation of benign epithelial cells, in situ lesions, and invasive epithelial cells in breast cancer slides

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15874702

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2017501456

Country of ref document: JP

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 20167026867

Country of ref document: KR

Kind code of ref document: A

REG Reference to national code

Ref country code: BR

Ref legal event code: B01A

Ref document number: 112016021802

Country of ref document: BR

NENP Non-entry into the national phase

Ref country code: DE

REG Reference to national code

Ref country code: BR

Ref legal event code: B01E

Ref document number: 112016021802

Country of ref document: BR

ENP Entry into the national phase

Ref document number: 112016021802

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20160922

122 Ep: pct application non-entry in european phase

Ref document number: 15874702

Country of ref document: EP

Kind code of ref document: A1