WO2024043190A1 - Inspection device, inspection system, and inspection method - Google Patents

Abstract

Provided are an inspection device, an inspection system, and an inspection method capable of accurately detecting a specific target object in an inspection target.　According to an embodiment, the inspection device includes an image acquiring unit and a processor. The image acquiring unit acquires captured image data including a captured image captured by emitting electromagnetic waves onto an inspection target, and physical property information representing a physical property of each portion of the captured image. The processor: detects a target object candidate, which is a candidate of a specific target object in the inspection target, on the basis of the physical property information included in the captured image data; detects a specific portion, which is a specific part or object in the inspection target; calculates a relative relationship between the target object candidate and the specific portion; and reports a target object candidate for which the calculated relative relationship satisfies a predetermined condition.

Description

Inspection equipment, inspection system and inspection method

Embodiments of the present invention relate to an inspection device, an inspection system, and an inspection method.

Conventionally, at an inspection site, it is sometimes necessary to perform inspection work to determine whether or not a predetermined object (hereinafter also referred to as a specific object) exists within the object to be inspected, such as baggage. When there are many packages that can be inspected, it takes a lot of time and effort for inspectors to open and inspect all the packages. Therefore, at an inspection site where a large number of packages are inspected, it is desirable to carry out open inspections on packages that are estimated to be likely to contain specific objects through various screening methods.

In recent years, an inspection system has been proposed that detects a specific object in a baggage by analyzing an image (X-ray image) taken of the baggage to be inspected using electromagnetic waves such as X-rays. Such inspection systems, for example, use X-rays to take images of baggage and determine whether the effective atomic number, which is the average atomic number, of the substances contained in each pixel that makes up the image is within a threshold. Detects what is presumed to be a specific object made of a specific substance. However, there is a problem in that it is difficult to distinguish between a specific object and an object having physical properties similar to the specific object using only the effective atomic number and density information obtained from an X-ray image.

Japanese Patent Application Publication No. 2009-92659

The problem to be solved by the present invention is to provide an inspection device, an inspection system, and an inspection method that can accurately detect a specific object in an inspection target.

According to the embodiment, the inspection device includes an image acquisition unit and a processor. The image acquisition unit acquires photographed image data including a photographed image photographed by irradiating the inspection object with electromagnetic waves and physical property information indicating physical properties of each part of the photographed image. The processor detects an object candidate that is a candidate for a specific object in the inspection target based on the physical property information included in the captured image data, detects a specific part that is a specific part or object in the inspection target, and detects a specific part that is a specific part or object in the inspection target. The relative relationship between the object candidate and the specific part is calculated, and the object candidate whose calculated relative relationship satisfies a predetermined condition is notified.

FIG. 1 is a diagram schematically showing the overall configuration of an inspection system including an inspection apparatus according to an embodiment. FIG. 2 is a block diagram showing a configuration example of an information management system including the inspection device according to the embodiment. FIG. 3 is a block diagram illustrating a configuration example of a control system in the imaging device and the inspection device of the inspection system according to the embodiment. FIG. 4 is a block diagram illustrating a configuration example of a higher-level management device in an information management system including the inspection device according to the embodiment. FIG. 5 is a flowchart for explaining the overall flow of inspection processing by the inspection apparatus according to the embodiment. FIG. 6A is a diagram illustrating an example of a photographed image photographed by a photographing device supplied to the inspection device according to the embodiment. FIG. 6(b) is a diagram illustrating an example of an image of an object candidate detected from a captured image by the inspection device according to the embodiment. FIG. 6C is a diagram showing an example of an image of a specific portion detected from a captured image by the inspection device according to the embodiment. FIG. 6D is a diagram illustrating an example of a target object candidate detected as a specific target object based on the relative relationship between the target object candidate detected by the inspection apparatus according to the embodiment and the identification unit. FIG. 7 is a flowchart for explaining an operation example of object candidate detection processing in the inspection apparatus according to the embodiment. FIG. 8 is a diagram illustrating an example of a two-dimensional image generated by slicing along each axis a three-dimensional image captured by an imaging device supplied to the inspection device according to the embodiment. FIG. 9 is a flowchart for explaining an operation example of relative relationship calculation processing in the inspection apparatus according to the embodiment. FIG. 10 is a diagram illustrating a display example of the inspection results of a specific object displayed on a display device by the inspection device according to the embodiment. FIG. 11 is a diagram illustrating a display example in which the inspection device according to the embodiment displays a detection result of a specific target object in a captured image. FIG. 12 is a diagram illustrating a display example in which the inspection device according to the embodiment displays the detection results of the specific portion in the captured image. FIG. 13 is a diagram illustrating an example of a display in which the inspection apparatus according to the embodiment displays the relative distance between the specific part and the target object candidate in a captured image. FIG. 14 is a diagram illustrating an example in which the inspection device according to the embodiment displays a detection result of a specific target object and also displays buttons for instructing display of target object candidates and specific parts. FIG. 15 is a diagram schematically showing an example in which the inspection apparatus according to the embodiment displays a two-dimensional image obtained by slicing a three-dimensional captured image at a designated location.

Embodiment

Embodiments will be described below with reference to the drawings.
FIG. 1 is a diagram for schematically explaining a configuration example of an inspection system 1 including an inspection apparatus 13 according to an embodiment.
The inspection system 1 according to the embodiment is a system for inspecting whether or not a specific detection object exists in a baggage to be inspected. Specific detection objects (specific objects) include, for example, dangerous goods, dangerous drugs, prohibited drugs, substances prohibited from being brought into or out of designated areas such as within Japan, etc. is assumed. Furthermore, the specific object to be detected does not have to be a solid having a specific shape, and includes substances such as liquids and powders.

In the configuration example shown in FIG. 1, the inspection system 1 includes a conveyor 11, an imaging device 12, an inspection device 13, a display device 14, an operating device 15, a speaker 16, and the like. The inspection device 13 is communicatively connected to the imaging device 12, the display device 14, the operating device 15, the speaker 16, and the like.

The conveyor 11 is a device that conveys the cargo M to be inspected. The conveyor 11 transports the baggage M to be inspected to the image capturing position (reading position) by the image capturing device 12 . For example, the conveyor 11 conveys a load M supplied by a worker. Furthermore, the conveyor 11 may be configured to transport the cargo M supplied by a robot arm or the like.

The photographing device 12 acquires photographed image data including a photographed image of the inspection object and physical property information indicating the physical properties of each part of the photographed image by irradiating electromagnetic waves to the baggage M to be inspected. The photographing device 12 supplies photographed image data of the baggage M to the inspection device 13. The imaging device 12 detects candidates (hereinafter also referred to as object candidates) and specific parts or objects (hereinafter also referred to as specific parts) that are presumed to be specific detection objects described later in the baggage M by the inspection device 13. Any method that can obtain photographed image data that can be used may be used. The photographing device 12 may be one that acquires two-dimensional image data as a photographed image, or may be one that acquires three-dimensional image data.

The imaging device 12 is, for example, an X-ray CT imaging device. An X-ray CT imaging device as an example of the imaging device 12 acquires three-dimensional X-ray image data as a photographed image by irradiating X-rays from around the baggage M conveyed by the conveyor 11. In addition, the X-ray CT imaging device as the imaging device 12 captures a three-dimensional X-ray image of the baggage M and the physical property information indicating the physical properties of each constituent unit (pixel or voxel) constituting the X-ray image. Get image data. An X-ray CT imaging device serving as the imaging device 12 supplies photographed image data acquired from the baggage M to the inspection device 13.

Note that the imaging device 12 is not limited to an X-ray CT imaging device, but in the embodiment described below, the description will be made assuming that the imaging device 12 is an X-ray CT imaging device. do.

The inspection device 13 has various functions, such as a function to process a photographed image of the baggage M taken by the photographing device 12 using electromagnetic waves.
For example, the inspection device 13 has a function (receiving section) for acquiring a photographed image of the baggage M taken by the photographing device 12 using electromagnetic waves, and displays output information based on information obtained by an inspection process described later on the display device 14 or a speaker. It has a function (transmission unit) to output using an output device such as 16.

In addition, the inspection device 13 has a function (object candidate detection unit) that detects a candidate (object candidate) that is estimated to be a specific object existing in the luggage M from the captured image data obtained from the imaging device 12. The inspection device 13 detects a candidate (object candidate) that is estimated to be a specific object in the photographed image of the baggage M obtained from the photographing device 12, based on setting values set based on the physical properties of the specific object. .

For example, the inspection device 13 detects object candidates based on the physical properties (density, effective atomic number, etc.) of each pixel or voxel in a captured image (X-ray image) captured by an X-ray CT imaging device as the imaging device 12. Note that a pixel corresponds to a pixel that is the smallest unit of two-dimensional image data. A voxel is the smallest unit of data that constitutes three-dimensional data, and represents a value in a regular grid unit. A voxel is a value corresponding to a pixel in two-dimensional image data.

In addition, the inspection device 13 has a function (specific part detection unit) of detecting a specific part or a specific object (hereinafter also simply referred to as a specific part) from an image taken of the baggage M. For example, the inspection device 13 detects a specific portion that is a specific region or a specific object in the captured image captured by the imaging device 12. The specific part is a part or object where a specific target object collected in advance is often hidden. The specific portion may be a portion or object having a predetermined shape, or may be a portion or object placed at a predetermined position in the luggage. For example, specific parts may include shoes, cameras, computers, walls of luggage such as trunks, and cigarettes.

The specific part detection unit of the inspection device 13 extracts an image of a region that is likely to be a specific part or object that has been set (learned) in advance from the photographed image taken by the imaging device 12, and recognizes the shape of the extracted image. Detects the specific part. Specifically, the specific part detection unit of the inspection device 13 can detect the specific part using semantic segmentation that associates labels and categories with all pixels in the captured image.

However, the method of detecting a specific part applied to the specific part detection unit is not limited to the above-mentioned method, and may be a method such as general image recognition or HOG (Histograms of Oriented Gradients) features extracted from an image. A method using object recognition such as SSD (Single Shot Detector, Single Shot MultiBox Detector) using machine learning or deep learning using feature amounts as input may also be used.

In addition, the inspection device 13 determines whether the object candidate is likely to be a specific object based on the relative relationship such as the distance between each object candidate and the specific part in the image taken by the imaging device 12 (if the object candidate is suspected to be a specific object). It has a function (relative relationship calculation unit) that evaluates that For example, when the target object candidate is adjacent to or included in the specific part, the inspection device 13 determines that there is a high possibility that the target object is the specific target object. In addition, the inspection device 13 calculates the relative distance between the object candidate and the specific part as a positional relationship, and if the calculated relative distance is less than or equal to a predetermined value, the possibility that the object candidate is a specific detection object is determined. may be determined to be high.

The display device 14 is an output device for notifying the inspector of the test results. The display device 14 displays a guidance screen and the like according to the control of the inspection device 13. The display device 14 displays a guide screen showing the results of the inspection process on the photographed image of the luggage M, etc., as a guide screen to be presented to the inspector. For example, the display device 14 displays an image clearly showing the object candidate and the specific portion in the image captured by the imaging device 12 and generated by the inspection device 13 .

The operating device 15 generates an operating signal according to the operating input of the inspector (operator), and supplies the operating signal to the inspection device 13. Further, the operating device 15 includes operating devices such as a keyboard and a pointing device. Further, the operating device 15 may be configured by a touch panel provided on the display screen of the display device 14 or the like.

The speaker 16 is an output device for notifying the inspector of test results and the like by voice. The speaker 16 outputs audio for notifying the inspector of audio guidance according to the results of the inspection process and the like.

In addition, the inspection device 13 has a function of executing a notification process to notify (issue a notification) information such as object candidates to the inspector. For example, the inspection device 13 displays, on the display device 14, object candidates that are determined to be highly likely to be specific objects, along with a photographed image taken by the photographing device 12 of the baggage M being conveyed by the conveyor 11.

In addition, the inspection device 13 displays all object candidates in the photographed image of the baggage M on the display device 14, and further distinguishes object candidates that are determined to be likely to be specific objects from other candidates. It may be displayed in a different color or mark. Furthermore, the inspection device 13 may display on the display device 14 images that clearly indicate the object candidate and the specific portion in the photographed image of the baggage M. Furthermore, the inspection device 13 may change the content displayed on the display device 14 in response to an instruction from a worker using the operating device 15.

Next, the configuration of the information management system 100 including the inspection system 1 according to the embodiment will be described.
FIG. 2 is a diagram showing a configuration example of an information management system 100 including the inspection system 1 according to the embodiment.
As shown in FIG. 2, the information management system 100 includes an upper management device 101 that is communicatively connected to the inspection device 13 of the inspection system 1 provided at each inspection site. The upper management device 101 functions as an information management device that collects data from the inspection devices in each inspection system 1 and supplies data to each inspection device.

The upper management device 101 is composed of, for example, a computer such as a server device. The upper management device 101 includes a storage device that stores information regarding tests performed by each test system 1. Further, the upper management device 101 may include an interface that connects to a server device that stores information regarding examinations.

The upper management device 101 acquires information from the inspection devices 13 in each inspection system 1. The upper management device 101 stores and aggregates information acquired from the inspection devices 13 of each inspection system 1. The upper management device 101 supplies information to the inspection devices 13 in each inspection system 1. For example, the upper management device 101 distributes setting values used for inspection processing to the inspection devices 13 of each inspection system 1. Further, the upper management device 101 may distribute update data for a program for each inspection device 13 to execute inspection processing.

Next, the configuration of the control system of the inspection device 13 in the inspection system 1 according to the embodiment will be explained.
FIG. 3 is a block diagram illustrating a configuration example of a control system of the inspection device 13 in the inspection system 1 according to the embodiment.
As shown in FIG. 3, the imaging device 12 includes an imaging section 21, a processing section 22, and an output section 23.
The imaging unit 21 shoots an image by irradiating an object to be imaged, such as a baggage M to be inspected, with electromagnetic waves such as X-rays. For example, when the imaging device 12 is an X-ray CT imaging device, the imaging unit 21 generates three-dimensional X-ray image data by emitting X-rays from around the baggage M to be inspected that is transported by the conveyor 11. get.

The processing unit 22 includes a processor, various types of memory, and the like, and the processor executes various processes by executing programs stored in the memory. For example, the processing unit 22 processes the image captured by the imaging unit 21 by irradiating electromagnetic waves, thereby generating a captured image that includes the captured image and physical property information indicating the physical properties of the constituent units (pixels or voxels) constituting the captured image. Generate data.

The output unit 23 is an interface that outputs data such as photographed image data. The output unit 23 includes an interface corresponding to the image interface 39 of the inspection device 13 and outputs captured image data to the inspection device 13. Further, the output unit 23 may be an input/output interface including an interface for inputting data such as control data from the connected inspection device 13.

Further, as shown in FIG. 3, the inspection device 13 includes a processor 31, a ROM 32, a RAM 33, a storage section 34, a communication section 35, a display interface (I/F) 36, an operation interface (I/F) 37, and an audio interface ( I/F) 38 and an image interface (I/F) 39.

The processor 31 executes arithmetic processing. The processor 31 is, for example, a CPU (Central Processing Unit). The processor 31 functions as a processing unit that executes various processes by executing programs stored in the ROM 32 or the storage unit 34 using the RAM 33.

The ROM 32 is a read-only nonvolatile memory. The ROM 32 stores program data, control data, and the like. The RAM 33 is a volatile memory that functions as a working memory. RAM 33 temporarily stores data.

The storage unit 34 is a rewritable nonvolatile memory. The storage unit 34 includes a hard disk drive (HDD), a solid state drive (SSD), and the like. The storage unit 34 stores information such as program data, setting values as control data, and test processing results.

The communication unit 35 is a communication interface for communicating with the upper management device 101. The processor 31 communicates with the upper management device 101 via the communication unit 35. The processor 31 transmits data such as processing results to the higher-level management device 101 via the communication unit 35, and receives data from the higher-level management device 101.

The display interface 36 is an interface for connecting to the display device 14 as an output device. The display interface 36 may be any interface that is compatible with the interface included in the display device 14. The processor 31 controls the display content displayed on the display device 14 via the display interface 36.

The operation interface 37 is an interface for connecting to the operation device 15. The operation interface 37 may be any interface as long as it corresponds to the interface included in the operation device 15. The processor 31 acquires information input by the operating device 15 via the operating interface 37 .

The audio interface 38 is an interface for connecting to the speaker 16 as an output device. The audio interface 38 may be any interface that is compatible with the interface included in the speaker 16. The processor 31 outputs audio from the speaker 16 as an output device via the audio interface 38.

The image interface 39 is an interface for connecting to the photographing device 12. The image interface 39 is an image acquisition unit (image acquisition interface) for acquiring a photographed image from the photographing device 12. The image interface 39 may be any interface as long as it is compatible with an interface included in the imaging device 12 such as an X-ray CT device. The processor 31 acquires, via the image interface 39, a photographed image (X-ray image) taken by an X-ray CT device serving as the photographing device 12. Furthermore, the processor 31 may control the photographing operation of the baggage M by the photographing device 12 via the image interface 39.

Next, the configuration of the upper management device 101 in the information management system 100 including the inspection system 1 according to the embodiment will be described.
FIG. 4 is a block diagram showing a configuration example of the upper management device 101 in the information management system 100 including the inspection system 1 according to the embodiment.
The upper management device 101 is an information management device that manages information about the entire inspection system 1 . The upper management device 101 is a computer that is communicatively connected to the inspection device 13 of the inspection system 1 provided at each inspection site. The upper management device 101 is configured by, for example, a server device.

In the configuration example shown in FIG. 4, the upper management device 101 includes a processor 41, a ROM 42, a RAM 43, a storage section 44, and a communication section 45.
Processor 41 executes arithmetic processing. The processor 41 is, for example, a CPU (Central Processing Unit). The processor 41 functions as a processing unit that executes various processes by executing programs stored in the ROM 42 or the storage unit 44 using the RAM 43.

The ROM 42 is a read-only nonvolatile memory. The ROM 42 stores program data, control data, and the like. RAM 43 is a volatile memory that functions as working memory. RAM 43 temporarily stores data.

The storage unit 44 is a rewritable nonvolatile memory. The storage unit 44 includes a hard disk drive (HDD), solid state drive (SSD), and the like. The storage unit 34 stores information such as program data, set values as control data, and data collected from each inspection device 13.

The communication unit 45 is a communication interface for communicating with the inspection device 13 in each inspection system 1. The processor 41 communicates with the inspection device 13 via the communication unit 45. The processor 41 receives data such as processing results from the inspection device 13 via the communication unit 45, and transmits data to the inspection device 13.

Next, an inspection process for inspecting the baggage M to be inspected in the inspection system 1 according to the embodiment will be described.
FIG. 5 is a flowchart for schematically explaining the flow of inspection processing in the inspection system 1 according to the embodiment. Further, FIGS. 6(a) to 6(d) are diagrams schematically showing examples of images obtained by each process of the inspection process.

In an inspection system 1 configured as shown in FIG. 1, packages M to be inspected are sequentially placed on a conveyor 11. The conveyor 11 transports the loaded baggage M to a position where an image is taken by the photographing device 12. The imaging unit 21 of the photographing device 12 acquires a photographed image showing the contents of the baggage M by irradiating electromagnetic waves onto the baggage M conveyed by the conveyor 11.

The photographed image may be image data that shows the condition inside the luggage M. For example, the X-ray CT device of the photographing device 12 acquires three-dimensional data indicating the state inside the baggage M as a photographed image by irradiating the baggage M transported to the photographing position with X-rays. Furthermore, physical property information indicating physical properties (density and effective atomic number) of each pixel or voxel constituting the photographed image is acquired.

The processing section 22 of the photographing device 12 generates photographed image data that includes a photographed image of the luggage M taken by the photographing section 21 and physical property information indicating the physical properties of each pixel or voxel in the photographed image. When the processing unit 22 of the photographing device acquires the photographed image data of the baggage M, the processing unit 22 outputs the photographed image data to the inspection device 13 through the output unit 23.

The inspection device 13 acquires photographed image data from the photographing device 12 through the image interface 39 (step S101). The processor 31 executes a process of acquiring captured image data as a process performed by the receiving section described above. For example, the inspection device 13 acquires photographed image data including a photographed image as shown in FIG. 6(a) and physical property information indicating the physical properties of each region (pixel or voxel) of the photographed image. The photographed image may be two-dimensional data, three-dimensional data, or a plurality of two-dimensional image data obtained by slicing three-dimensional data along a specific axis. . In the following explanation, in order to simplify the explanation, it is assumed that the photographed image is two-dimensional image data.

The processor 31 of the inspection device 13 executes an object candidate detection process of detecting an object candidate as a specific object candidate from the captured image data acquired from the imaging device 12 (step S102). The processor 31 executes object candidate detection processing as processing by the object candidate detection section described above. For example, the processor 31 selects locations (pixels) whose physical properties are similar or consistent with the physical properties of the specific target in the photographed image as object candidates based on setting values that are set according to the physical properties of the specific target to be detected. To detect. FIG. 6(b) is a diagram showing an example of object candidates detected from the captured image data as shown in FIG. 6(a).

In addition, the processor 31 of the inspection device 13 detects, based on the photographed image data acquired from the photographing device 12, a specific part or object in the photographed image of the baggage M that is a place where a specific object to be detected is likely to be placed. A detection process for detecting a specific part (specific part) is executed (step S103).

The processor 31 executes a specific part detection process as the process by the specific part detection unit described above. For example, the processor 31 sets in advance the shape of a region or object to be detected as a specific part, and detects, as a specific part, an image area having a shape similar to the shape of the specific part to be detected in the photographed image. FIG. 6(c) is a diagram showing an example of a specific part detected from the captured image data as shown in FIG. 6(a).

After executing the object candidate detection process and the identification part detection process, the processor 31 executes a relative relationship calculation process that calculates the relative relationship between the detected object candidate and the identification part (step S104). The processor 31 executes a relative relationship detection process as a process by the above-mentioned relative relationship calculation unit. The relative relationship between the object candidate and the identification unit is information for evaluating whether or not the object candidate is suspected to be the specific object to be detected. The processor 31 calculates, for example, the relative distance between the object candidate and the specific part as a relative relationship.

Once the processor 31 calculates the relative relationship between the target object candidate and the specific part, it detects the target object candidate suspected to be the specific target based on the relative relationship with the specific part (step S105). For example, the processor 31 detects an object candidate that is suspected of being a specific object (detects it as a specific object) that satisfies a predetermined condition in which the relative relationship (for example, relative distance) with the specific part is a preset value. ) is determined to be a thing.

The processor 31 notifies (announces) the detection result of the object candidate that is suspected to be the specific object based on the relative relationship between the object candidate and the identification unit using the display device 14 and the speaker 16 as output devices. Alert processing is executed (step S106). For example, when the processor 31 detects an object candidate that is suspected to be a specific object as a process performed by the above-mentioned transmitting unit, the processor 31 issues an alarm to notify the detection result through an output device such as the display device 14 or the speaker 16. Execute processing.

For example, the processor 31 causes the display device 14 to display object candidates whose relative distance to the specific portion is less than or equal to a preset threshold as being suspected to be the specific object. FIG. 6(d) shows object candidates that are determined to be suspected to be specific objects based on the relative relationship between the object candidates shown in FIG. 6(b) and the identification portion shown in FIG. 6(c). FIG. 4 is a diagram showing an example of a display displaying a specific part associated with the target object candidate.

The baggage for which the detection result of the target object candidate is displayed on the display device 14 through the notification process is inspected by the inspector. The inspector performs inspection work to inspect the contents of the baggage while referring to information such as the detection results displayed by the notification process. Note that the inspection system 1 may perform inspection work on all packages M, or may perform inspection work on packages M in which an object candidate suspected of being a specific object has been detected. It is also possible to carry out the operation.

Next, an example of object candidate detection processing performed by the object candidate detection unit of the inspection device 13 according to the embodiment will be described.
FIG. 7 is a flowchart for explaining an example of object candidate detection processing performed by the object candidate detection unit of the inspection device 13 according to the embodiment.

The processor 31 of the inspection device 13 acts as an object candidate detection unit, based on, for example, the photographed image data of the luggage M obtained from the photographing device 12 and the effective atomic number and density data of each part in the photographed image. An object candidate that is similar to or coincides with the physical properties of a specific object to be detected is detected. Specifically, whether the effective atomic number and density of each pixel or voxel of the captured image, which is the input data, are within the range of a threshold determined based on the physical property values of the specific target to be detected collected in advance. It is determined whether or not the object is a candidate object.

In the processing example shown in FIG. 7, the processor 31 of the inspection device 13 acquires the physical property value Z of the specific object to be detected (step S201). Here, the physical property value Z of the specific object may be input by a manager or an inspector using the operating device 15, or may be obtained from an external device such as the upper management device 101.

Upon acquiring the physical property value Z of the specific target object, the processor 31 sets a threshold value E based on the physical property value Z of the specific target object (step S202). For example, the processor 31 sets a threshold value E that sets a predetermined tolerance range for the physical property value Z of the specific object.

Note that the processes in steps S201 and S202 may be replaced with a process in which the administrator or inspector sets the threshold value E using the operating device 15. Further, in the processing of steps S201 and S202, information specifying the threshold E is obtained from the upper management device 101, and the threshold E obtained from the upper management device 101 is set as a setting value in the object candidate detection process. Also good.

The processor 31 of the inspection device 13 that has set the threshold value E, which is a set value in the object candidate detection process, uses the photographic image taken by the photographing device 12 of the baggage M to be inspected using electromagnetic waves such as X-rays, and the photographed image. Captured image data including physical property information indicating the physical properties of each part is acquired (step S203). Upon acquiring the photographed image data of the luggage M from the photographing device 12, the processor 31 executes a process of detecting object candidates included in the photographed image (steps S204 to S207).

In the processing example shown in FIG. )-Z|<E) (step S205). If |I(p)−Z|<E (step S205, YES), the processor 31 selects pixel p in the photographed image as an object candidate (step S206). The processor 31 repeatedly executes the processes of steps S205 and S206 for all pixels forming the photographed image of the luggage M (steps S204, S207).

When the processing of steps S205 and S206 for all pixels in the photographed image is completed, the processor 31 detects, as a target object candidate, an image area consisting of a set of pixels detected as target object candidate pixels (step S208). For example, the processor 31 selects a pixel from an image region in which pixels having a physical property value I(p) whose difference from the physical property value Z of the specific object is less than a threshold value E (or from a set of pixels whose relative distance is less than or equal to a predetermined distance) image area) is detected as a target object candidate.

According to the example of the target object candidate detection processing described above, the inspection device detects the physical property values such as the effective atomic number and density data of each part in the photographed image obtained by irradiating electromagnetic waves such as an X-ray image, and the specific target object. An object candidate is detected based on whether the difference between the object and the physical property value is within a predetermined threshold. As a result, the inspection device according to the embodiment can detect candidates for the specific object present in the baggage, even if the specific object to be detected is a substance such as a liquid or powder that does not have a specific shape. can.

In addition, in the above-mentioned object candidate detection processing, a two-dimensional image is used as the processing target, and it is determined whether or not each pixel is an object candidate. However, if the photographed image is three-dimensional data, the processing is performed for each voxel. An object candidate in a three-dimensional photographed image may be detected by determining whether or not the object is an object candidate.

In addition, the processor 31 of the inspection device 13 performs the above-mentioned object candidate detection process on a plurality of two-dimensional images obtained by slicing (dividing) a three-dimensional image as a captured image along each axis. You may also choose to execute it.
FIG. 8 is a diagram showing an example of a plurality of two-dimensional images obtained by slicing (dividing) three-dimensional data as a captured image along each axis (x-axis, y-axis, z-axis).
When a plurality of two-dimensional image data as shown in FIG. 8 are obtained, the processor 31 executes the object candidate detection process as described above for each two-dimensional image data. After detecting the object candidate in each two-dimensional image data, the processor 31 may detect the object candidate in the three-dimensional space by superimposing the detection results of the object candidate in each two-dimensional image.

Furthermore, the process for detecting object candidates is not limited to the method described above, but may be any method that can detect object candidates using data on specific objects collected in advance. For example, as the object candidate detection process, a hash table, machine learning, or the like may be used to detect object candidates. Further, as long as the specific target object has a feature as a shape, general object recognition may be used to detect the target object candidate.

Next, relative relationship calculation processing by the relative relationship calculation unit of the inspection device 13 according to the embodiment will be described in detail.
The processor 31 of the inspection device 13, as a relative relationship calculation unit, calculates the relative relationship between the target object candidate detected from the captured image and the specific unit. The relative relationship is calculated as an index value for evaluating whether the object candidate is a specific object to be detected. For example, as the relative relationship, the distance (relative distance) between the target object candidate and the specific part is calculated. In this case, it can be determined that the object candidate that is close to the specific part is suspected to be the specific object.

Furthermore, as the relative relationship, an inclusive relationship between the target object candidate and the specific part, or a positional relationship such as whether they are adjacent or not, may be calculated. In this case, the object candidate within the specific portion or the object candidate adjacent to the specific portion can be evaluated as being suspected to be the specific object.

Further, as a relative relationship, physical property information (for example, average values of density, effective atomic number, etc.) for a combination of physical property information of the target object candidate and physical property information of the specific part may be calculated. In this case, an object candidate suspected of being a specific object can be detected even in a state where the substance forming the identification portion and the substance forming the object candidate coexist.

FIG. 9 is a flowchart for explaining the process of calculating the relative distance between the object candidate and the specific part as an example of the process of calculating the relative relationship by the relative relationship calculation unit of the inspection apparatus 13 according to the embodiment.
In FIG. 9, a processing example will be described in which the relative distance between the target object candidate and the specific portion is calculated based on the distance between each pixel detected as the target object candidate and each pixel forming the specific portion. An example of calculating the relative distance as the relative relationship between the object candidate and the specific portion is a method of calculating the Euclidean distance between the center position of the object candidate and the center position of the specific portion. However, the distance between the center positions is greatly influenced by the shape of the object candidate or the specific part. For example, if the region of the object candidate (or specific portion) has an L-shape, the center position of the region may not be on the object candidate (or specific portion). Therefore, in the processing example shown in FIG. 9, the relative distance between the target object candidate and the specific part is calculated based on the distance between each pixel of the target object candidate and each pixel of the specific part.

That is, the processor 31 of the inspection device 13 acquires information indicating the area of the object candidate T and the area of the specific part W detected in the photographed image of the baggage M to be inspected (step S301). Here, it is assumed that the processor 31 acquires the image area of the target object candidate T and the image area of the specific part W for which the correlation is to be calculated. However, if a plurality of object candidates or a plurality of specific parts exist in the captured image, the processor 31 performs the correlation by executing the processes of steps S301 to S307 for all combinations of each object candidate and each specific part. Let us calculate the relationship (relative distance).

Upon acquiring the information indicating the image area of the specific portion W in the captured image, the processor 31 sets the total number of pixels forming the image of the specific portion W as the number of loops depending on the specific portion (step S302). Further, upon acquiring information indicating the image area of the target object candidate T in the photographed image, the processor 31 sets the total number of pixels forming the image of the target object candidate T as the number of loops according to the target object candidate (step S303 ).

The processor 31 calculates the distance between each pixel of the target object candidate and each pixel of the specific part, and uses the calculated distance between each pixel as a distance for calculating the relative distance between the specific part W and the target object candidate T. The information is stored in the RAM 33 or the storage unit 34 (step S304). The processor 31 repeatedly executes the process of calculating the distance of each pixel of the specific part to one pixel of the target object candidate for the number of pixels of the specific part (the number of loops depending on the specific part) (step S305).

When the calculation of the distance of each pixel of the specific part to one pixel of the target object candidate is completed, the processor 31 executes a process of calculating the distance of each pixel of the specific part to the next pixel selected from the target object candidate. do. The processor 31 repeatedly executes the process of calculating the distance of each pixel of the specific part to the pixels sequentially selected from the target object candidates for the total number of pixels of the target object candidates (the number of loops according to the target object candidates). Step S306). Through the processing in steps S302 to S306, the processor 31 calculates the distances between all pixels in the specific part and all pixels in the object candidate.

When the calculation of the distance between each pixel of the target object candidate T and each pixel of the specific part W is completed, the processor 31 calculates the distance between the target object candidate T and the specific part W based on the distance between each pixel stored in the RAM 33 or the storage part 34. A relative distance as a relative relationship with W is calculated (step S307). For example, the processor 31 may calculate the distance that is the minimum value from the distances calculated for the number of combinations of each pixel of the target object candidate T and each pixel of the specific portion W as the relative relationship.

However, in reality, noise may be mistakenly detected as a specific part. Noise that is mistakenly detected as a specific part may also be detected at a position close to the target object candidate. In such a case, if there is even a small amount of noise in a location closer to the target object candidate than the actual specified part, the minimum distance to the target object candidate will be calculated as a value smaller than the actual distance to the specified part. be done.

For this reason, the processor 31 calculates the distance between each pixel detected as a target object candidate and each pixel of the specific part as the relative relationship (relative distance) between the target object candidate and the specific part, and then calculates the minimum value. It is also possible to take a percentile value instead of . By calculating the correlation that indicates the distance between each pixel of the object candidate and each pixel of the specific part using percentile values, it is possible to confirm that the object candidate is likely to be a specific object even if there is noise detected as the specific part. This enables robust evaluation.

Note that in the processing example shown in FIG. 9, the distances of all pixels in the specific part are calculated for all pixels in the target object candidate, but in order to reduce the amount of calculation, the distances are calculated for all combinations. There is no need to calculate the distance. For example, the processor 31 may reduce the number of combinations of pixels for which distances are calculated by calculating distances for combinations of pixels sampled from the object candidate and pixels sampled from the specific portion. By reducing the number of combinations of pixels for which distances are calculated in the processing example shown in FIG. 9, the amount of calculation in the relative relationship calculation process can be reduced.

In addition, in the example of the correlation calculation process described above, the distance between pixels is calculated using a two-dimensional image as the processing target, but if the captured image is three-dimensional data, each voxel in the object candidate and the What is necessary is to calculate the distance to each voxel in the area. Thereby, even if the photographed image is three-dimensional data, it is possible to calculate the relative distance as the relative relationship between the object candidate and the specific part.

Next, a process for reporting detection results based on the relative relationship of the inspection apparatus 13 according to the embodiment will be described in detail.
The processor 31 of the inspection device 13 executes a notification process of notifying the detection result of a specific target object based on the relative relationship using an output device. For example, the processor 31 of the inspection device 13 displays a display screen of the inspection results for the specific object in which an image area suspected to be the specific object (an image area detected as the specific object) in the photographed image of the luggage M is highlighted. is displayed on the display device 14.

FIG. 10 is a diagram illustrating a display example in which the inspection device 13 displays the inspection results of the specific object in the baggage M on the display device 14 as a notification process.
The display example shown in FIG. 10 displays a photographed image of baggage M taken by the photographing device 12, and objects that are determined to be suspected of being a specific object based on the relative relationship with a specific part on the photographed image. This is an example of a guidance display in which an image region of an object candidate (an image region detected as a specific target object) is highlighted.

The image area suspected to be a specific object may be highlighted so that it can be visually distinguished from other image areas. For example, an image area suspected of being a specific object may be highlighted by displaying it in a different color from other areas. Further, an image area suspected of being a specific object may be displayed in a highlighted manner by surrounding it with a square, an ellipse, or the like.

In addition, the results of the inspection process that the inspection device 13 displays on the display device 14 through the alarm process include not only the image area of the target object candidate that has been determined to be suspected of being the specific target object, but also the specific part, the target object Candidates or information indicating relative relationships may be displayed.

FIG. 11 is a table displaying information indicating object candidates detected in the object candidate detection process as well as image areas determined to be suspected to be specific objects in the photographed image (specific object detection results). It is a figure which shows an example.
In the display example shown in FIG. 11, image areas of object candidates other than object candidates determined to be suspected of being specific objects are displayed so that they are surrounded by elliptical guide lines so that the inspector can visually recognize them. This is an example. Further, the area detected as a target object candidate may be displayed so as to be surrounded by a rectangle or the like, or may be displayed in a specific color set as the area of the target object candidate.

FIG. 12 is a diagram illustrating a display example in which a guide indicating a specific part detected by the specific part detection process is displayed together with an image area determined to be suspected of being a specific target object in a photographed image.
The display example shown in FIG. 12 is an example in which a rectangular guide line surrounds the image area of the specific part so that the inspector can visually recognize the image area detected as the specific part in the photographed image. Further, the image area detected as the specific portion may be displayed so as to be surrounded by an ellipse or the like, or may be displayed in a specific color set as the area of the target object candidate.

FIG. 13 is a diagram illustrating a display example in which information indicating the relative relationship (relative distance) of each object candidate with respect to the specific portion is displayed together with an image area determined to be suspected of being a specific object in a photographed image. .
In the display example shown in FIG. 13, information (number of pixels) indicating the relative distance as the relative relationship of each object candidate to the part detected as the specific part is displayed. Alternatively, object candidates and specific parts may be displayed in the captured image, and information such as density and effective atomic number in the image area of the object candidates and specific parts may be displayed side by side (or overlapping).

FIG. 14 is a diagram illustrating a display example in which a detection result of a specific target object is displayed and buttons for instructing display of target object candidates and specific parts are provided.
In the display example shown in FIG. 14, a button for instructing display of all object candidates, a button for instructing display of a specific part, and a button for selecting individual object candidates are displayed. When a button instructing the display of object candidates is input through the operating device 15, images of all object candidates detected as object candidates are displayed on the display device 14, as shown in FIG. .

Furthermore, when a button for instructing display of the specific portion is input using the operating device 15, an image of the area detected as the specific portion is displayed on the display device 14. Further, when a button for selecting an individual object candidate is input through the operating device 15, the display device 14 displays an image of the selected object candidate as well as information regarding the object candidate (for example, information indicating physical properties, etc.). Alternatively, information indicating the relative relationship with the specific part, etc.) is displayed.

Furthermore, if the captured image is three-dimensional data, the processor 31 of the inspection device 13 may display the captured image on the display device 14 in 3D. In this case as well, the processor 31 may display the area of the object candidate detected as the specific object in a different color from the other areas. Further, even when displaying a photographed image in 3D, the processor 31 may display object candidates, specific parts, etc. on the photographed image according to instructions from the inspector. Furthermore, when displaying a photographed image as a 3D image on the display device 14, the processor 31 rotates, enlarges, reduces, or moves the 3D photographed image in accordance with operations by the inspector. It's okay.

Furthermore, when the captured image is three-dimensional data, the processor 31 may display on the display device 14 a two-dimensional image sliced at a location (plane) designated by the inspector.
FIG. 15 is a diagram schematically showing an example of displaying a two-dimensional image obtained by slicing a three-dimensional captured image at a designated location.
According to the example shown in FIG. 15, the inspection device 13 displays the captured image as a two-dimensional image obtained by slicing the three-dimensional captured image at a designated location on the display device 14, and further displays an object on the captured image. Object candidates, specific parts, areas detected as specific objects, etc. are displayed. The location where the three-dimensional photographed image is sliced may be a predetermined location set in advance, or a location designated by the inspector using the operating device 15.

In addition, in the above description, an example of the operation of the inspection system 1 in which the imaging device 12 and the inspection device 13 are configured separately has been described, but the processing of the inspection device 13 described above is configured to be executed by the imaging device 12. It's okay. That is, in the above-described inspection system 1, the above-described photographing device 12 and inspection device 13 may be realized by an integrated device.

In addition, in the above example, the operation example of the inspection device 13 was mainly explained, but the information management system 100 shown in FIG. The information may be collected by the upper management device 101. That is, information indicating various processing results and inspection work results by the inspection device 13 may be transmitted from the inspection device 13 to the higher-level management device 101 and stored in the higher-level management device. As a result, the upper management device 101 analyzes the information collected from the inspection devices 13 of the inspection system 1 located in various places, and changes the parameters for various processes executed by each inspection device 13 according to the analysis results. It's okay.

As a specific example, the upper management device collects information indicating the parts and objects where specific target objects are detected during actual inspection work from each inspection device, and sets each inspection to detect those parts and objects as specific parts. Parameters for the detection process of the specific unit may be set in the device. In addition, the upper management device calculates statistics such as the probability (accuracy) that an object candidate detected as a specific object will be confirmed as a specific object in actual inspection work from the information collected from each inspection device, and calculates the statistics. The setting value for determining a target object candidate as a specific target object may be adjusted based on the relative relationship with the specific part according to the statistical amount of .

As described above, the inspection apparatus according to the embodiment acquires photographed image data including a photographed image photographed by irradiating an inspection target with electromagnetic waves and physical property information on each part of the photographed image. The inspection device detects the target object candidate and the specific part in the photographed image, and notifies the target object candidate that is suspected to be the specific target based on the relative relationship between the target object candidate and the specific part.

As a result, the inspection device according to the embodiment detects a specific part where a specific target object is likely to be placed, and identifies an area suspected of being a specific target object based on the relative relationship between the specific part and the candidate object. be able to. As a result, it is possible to narrow down object candidates that are difficult to distinguish from the specific object based on the physical property values obtained by irradiating the inspection object with electromagnetic waves based on the relative relationship with the specific part, and identify the specific object with high accuracy. It is possible to detect areas where a problem is suspected.

Although several embodiments of the invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included within the scope and gist of the invention, as well as within the scope of the invention described in the claims and its equivalents.

Claims (12)

1. an image acquisition unit that acquires photographed image data including a photographed image photographed by irradiating an inspection target with electromagnetic waves and physical property information indicating physical properties of each part of the photographed image;
   detecting an object candidate that is a candidate for a specific target in the inspection target based on physical property information included in the photographed image data; detecting a specific part that is a specific part or object in the inspection target; a processor that calculates a relative relationship between an object candidate and the specific unit and notifies an object candidate for which the calculated relative relationship satisfies a predetermined condition;
   An inspection device with
2. The processor calculates whether the object candidate and the specific part are in an adjacent or inclusive relationship as the relative relationship, and notifies the object candidate adjacent to or included in the specific part.
   The inspection device according to claim 1.
3. The processor calculates a relative distance between the target object candidate and the specific unit as the relative relationship, and notifies target object candidates whose relative distance to the specific unit is equal to or less than a predetermined threshold.
   The inspection device according to claim 1.
4. The processor calculates a distance between each constituent unit of the target object candidate and each constituent unit of the specific part, and calculates the minimum value of the calculated distances as a relative distance between the target object candidate and the specific part. and
   The inspection device according to claim 3.
5. The processor calculates a distance between each constituent unit of the object candidate and each constituent unit of the identification section, and calculates a predetermined percentile value for the calculated distance between the object candidate and the identification section. Let be the relative distance of
   The inspection device according to claim 4.
6. The processor causes a display device to display, together with the photographed image, a position where an object candidate whose relative relationship with the specific portion satisfies the predetermined condition is present.
   The inspection device according to claim 1.
7. The processor further causes a display device to display information indicating all object candidates detected from the photographed image.
   The inspection device according to claim 6.
8. The processor further causes a display device to display information indicating the specific portion detected from the photographed image.
   The inspection device according to claim 6.
9. The photographed image is an X-ray image obtained by photographing the inspection object by irradiating X-rays.
   An inspection device according to any one of claims 1 to 8.
10. The photographed image is three-dimensional data obtained by photographing the inspection object using an X-ray CT device, and the processor determines the relative relationship between the object candidate and the specific portion in the three-dimensional data as the photographed image. calculate,
    The inspection device according to claim 9.
11. In an inspection system having an imaging device and an inspection device,
    The photographing device is
    an imaging unit that captures a captured image by irradiating the inspection target with electromagnetic waves;
    a processing unit that generates photographed image data including the photographed image and physical property information indicating physical properties of each part of the photographed image;
    an output unit that outputs the photographed image data;
    The inspection device includes:
    an image acquisition unit that acquires photographed image data output from the photographing device;
    detecting an object candidate that is a candidate for a specific target in the inspection target based on physical property information included in the photographed image data; detecting a specific part that is a specific part or object in the inspection target; a processor that calculates a relative relationship between an object candidate and the specific unit and notifies an object candidate for which the calculated relative relationship satisfies a predetermined condition;
    Inspection system.
12. An inspection method for inspecting a specific object in an inspection target,
    Obtaining photographed image data including a photographed image taken by irradiating an inspection target with electromagnetic waves and physical property information indicating physical properties of each part of the photographed image,
    detecting an object candidate that is a candidate for a specific object in the inspection target based on physical property information included in the photographed image data;
    detecting a specific part that is a specific part or object in the inspection target;
    Calculating the relative relationship between the object candidate and the specific part,
    Notifying object candidates for which the calculated relative relationship satisfies a predetermined condition;
    Inspection method.
    
    

Images