JP2021105577A - Learning model construction device, learning model construction method, and learning model construction program - Google Patents

Abstract

To construct a learning model with which it is possible to correct a difference in coloring processing on transparent images between manufacturers and obtain accurate determination results as to contents, etc.SOLUTION: Color correction information for the occurrence ratio of each pixel value in a histogram of hues for other models and a histogram of image information of saturation and luminance per prescribed angle in a hue circle to be precisely matched to the occurrence ratio of each pixel value in a histogram of hues for a main model and a histogram of image information of saturation and luminance per prescribed angle in a hue circle is created. This color correction information for hue, saturation and luminance is converted into color correction information for red, green and blue colors, and the color tone of the image information for other models is converted into the color tone of the image information for the main model by the color correction information for red, green and blue colors. Meanwhile, the image information for other models having been converted into the color tone of the image information for the main model is cumulatively stored in a storage unit as learning images for the main model for object determination.SELECTED DRAWING: Figure 16

Description

The present invention relates to a learning model construction device, a learning model construction method, and a learning model construction program.

Today, for example, at airports, factories, etc., there are known object inspection devices capable of visually recognizing stored items or internal structures without opening or destroying luggage or products. In such an object inspection apparatus, as disclosed in Patent Document 1 (Japanese Unexamined Patent Publication No. 11-230918), a fluoroscopic image captured by X-rays is used as an example.

The fluoroscopic image is colored according to the attenuation rate (depending on the atomic number) of the X-rays applied to the object to be inspected and the density of the object. Further, a learning model in which the stored object or the internal structure is learned is generated based on the plurality of fluoroscopic images. Then, based on this learning model, the content or internal structure shown in the fluoroscopic image captured at the time of inspection is determined and the determined image is displayed. The inspector judges the contents or the internal structure of the baggage or the product based on this judgment image, and performs an open inspection or the like as necessary.

Japanese Unexamined Patent Publication No. 11-230918

Many fluoroscopic images are required to generate a learning model with high judgment accuracy. Therefore, it is conceivable to increase the number of fluoroscopic images for learning by adding not only the fluoroscopic images taken by oneself but also the fluoroscopic images taken by a third party to the fluoroscopic images for learning.

However, in the object inspection devices used today, the coloring process for the fluoroscopic image differs depending on the manufacturer of the object inspection device. Therefore, the hue of the created learning model also differs depending on the manufacturer of the object inspection device that has captured the original fluoroscopic image. Therefore, for example, if the fluoroscopic image captured by the object inspection device manufactured by company A and the fluoroscopic image of the object inspection device manufactured by company B are used to generate a learning model with the same hue, learning with low determination accuracy. There was a problem that the model was generated.

The present invention has been made in view of the above-mentioned problems, and is a construction of a learning model that makes it possible to obtain an accurate determination result of the contents and the like by correcting the difference in the coloring process between the manufacturers for the fluoroscopic image. The purpose is to provide a device, a learning model construction method, and a learning model construction program.

In order to solve the above-mentioned problems and achieve the object, the present invention presents the first image information of the first image format as the second of the second image format including the hue information, the saturation information and the brightness information. A format conversion unit that converts to image information of the above, and a histogram creation unit that divides hue information at predetermined angles on the hue circle and creates a histogram of hue information, saturation information, and brightness information for each divided angle. , Hue information corresponding to the first hue, hue information corresponding to the second hue different from the first hue, and saturation information in the appearance rate of each pixel value shown in the histogram of the saturation information and the brightness information. And a color correction information creation unit that creates color correction information of hue information, saturation information, and brightness information for matching the appearance rate of each pixel value shown in the histogram of brightness information, hue information, saturation information, and A conversion unit that converts each color correction information of the brightness information into color correction information corresponding to the first image format, and the first image information of the first image format of the second hue different from the first hue. , The color conversion unit that converts each color correction information converted by the conversion unit into the first image information of the first hue and the first image information converted into the first hue are used for object determination. It has a learning unit that is stored and stored in a storage unit as the first image information of the above.

INDUSTRIAL APPLICABILITY According to the present invention, it is possible to construct a learning model capable of obtaining an accurate determination result of a content or the like by correcting a difference in coloring process between manufacturers for a fluoroscopic image.

FIG. 1 is a system configuration diagram of an object inspection system according to an embodiment. FIG. 2 is a diagram showing a hardware configuration of an object inspection device provided in the object inspection system according to the embodiment. FIG. 3 is a diagram showing a hardware configuration of an analysis device provided in the object inspection system according to the embodiment. FIG. 4 is a functional block diagram of each function realized by software when the color correction table creation unit executes the color correction table creation program. FIG. 5 is a diagram showing an outline of a process of creating a color correction table for color correction of another model image. FIG. 6 is a diagram showing an outline of a determination operation for determining the contents of the luggage shown in the different model image based on the different model image color-corrected by the color correction table for color correction of the different model image. FIG. 7 is a flowchart showing the flow of the color correction table creation operation. FIG. 8 is a diagram showing an arithmetic expression used when converting an RGB value into an HLS value. FIG. 9 is a diagram showing an arithmetic expression used when converting an RGB value into an HSV value. FIG. 10 is a diagram for explaining the operation of creating a color correction table for hue H. FIG. 11 is a diagram for explaining an operation of creating a color correction table of luminance L and saturation S for each fixed width of hue H. FIG. 12 is a diagram for explaining the inconvenience caused by the existence of a portion where the number of pixels is concentrated in the histogram. FIG. 13 is a diagram for explaining the dispersion processing of the histogram in the color correction table forming unit. FIG. 14 is a diagram showing the characteristics of the hue H and the luminance value V in the HSV space. FIG. 15 is a flowchart showing a flow of a determination operation of the contents of an object or the like based on an image of another model that has been color-converted by a color correction table. FIG. 16 is a flowchart showing a flow of a color correction table forming operation for correcting the color tone of the main model image to the color tone of another model image. FIG. 17 is a flowchart showing a flow of a determination operation in which a color tone of a main model image is converted into a color tone of another model image in an object inspection device serving as a main model to determine the contents of an object or the like. FIG. 18 is a schematic diagram showing a learning operation of the main model learning image. FIG. 19 is a diagram showing a determination operation of the contents and the like based on the main model learning image. FIG. 20 is a schematic diagram showing a learning operation of another model learning image. FIG. 21 is a diagram showing a determination operation of the contents and the like based on the learning image of another model.

Hereinafter, the object inspection system of the embodiment to which the learning model construction device of the present invention, the learning model construction method, and the learning model construction program is applied will be described with reference to the drawings.

(System configuration)
FIG. 1 is a system configuration diagram of the object inspection system of the embodiment. As shown in FIG. 1, the object inspection system includes an object inspection device 1 and an analysis device 2. The object inspection device 1 and the analysis device 2 are connected to each other by wire or wirelessly.

The object inspection device 1 and the analysis device 2 may be connected via a public network such as the Internet or a private network such as a LAN. Further, in this example, the object inspection device 1 and the analysis device 2 will be described as separate devices, but the object inspection device 1 is provided with the function of the analysis device 2 and is physically regarded as one device (object inspection device). May be good.

The object inspection device 1 has a main body 16, an inlet 16a of the luggage 4, and an exit 16b of the luggage 4. Further, the object inspection device 1 has a monitor device 14. The main body 16 forms a fluoroscopic image by irradiating the luggage 4 with, for example, X-rays, while moving the luggage 4 carried in from the inlet 16a to the outlet 16b by the roller conveyor. This fluoroscopic image is transmitted to the analysis device 2.

The analysis device 2 stores a learning image for discriminating a stored object or the like inside the luggage 4 by learning a plurality of fluoroscopic images formed by the object inspection device 1. When the fluoroscopic image is supplied from the object inspection device 1, the analysis device 2 determines the stored items inside the luggage 4 shown in the fluoroscopic image based on the stored learning image. Then, the determination image that is the determination result is supplied to the monitoring device 14 of the object inspection device 1. As a result, a determination image showing the stored items inside the luggage 4 can be displayed on the monitor device 14.

The inspector determines the stored items of the luggage 4 based on the determination image, and performs an open inspection or the like as necessary.

(Hardware configuration of object inspection device)
FIG. 2 is a diagram showing a hardware configuration of the object inspection device 1. As shown in FIG. 2, the object inspection device 1 has an X-ray source 21 that irradiates the luggage 4 with X-rays, and a roller conveyor 22 that moves the luggage 4 carried in from the inlet 16a to the outlet 16b. There is. Further, the object inspection device 1 forms an image (RGB image) in an RGB format corresponding to the X-ray detector 23 for detecting the X-ray transmitted through the luggage 4 and the X-ray detection output of the X-ray detector 23. It has a processing unit 24. The fluoroscopic image (RGB image) is supplied to the analysis device 2 as described above, and is used for determining an internal storage object or the like. Further, the object inspection device 1 has a monitor device 14 that displays a determination image from the analysis device 2.

Of the hardware shown in FIG. 2, some hardware such as the image processing unit 24 may be realized by software.

(Hardware configuration of analyzer)
FIG. 3 is a diagram showing a hardware configuration of the analysis device 2. As shown in FIG. 3, the analysis device 2 includes an acquisition unit 31, an image learning unit 32, a color correction table creation unit 33, a storage unit 34, a color correction unit 35, and a determination unit 36.

The acquisition unit 31 acquires a fluoroscopic image (main model image) formed by the main model, which is an object inspection device manufactured by a predetermined manufacturer. Further, the acquisition unit 31 acquires a fluoroscopic image (another model image) formed by another model manufactured by a manufacturer different from the manufacturer that manufactures such a main model.

The image learning unit 32 learns a plurality of main model images, and stores a learning image (main model learning image) for discriminating the stored items inside the luggage 4 in the storage unit 34. This main model learning image is added each time it is learned and stored in the storage unit 34. Further, this main model learning image is used when creating a color correction table for correcting the hue of the perspective image of another model to the hue of the perspective image of the main model.

Further, the image learning unit 32 learns a plurality of images of different models, and stores the learning images (learning images of different models) for discriminating the stored items inside the luggage 4 in the storage unit 34. This different model learning image is added each time it is learned and stored in the storage unit 34. Further, this different model learning image is used when creating a color correction table for correcting the hue of the perspective image of the main model to the hue of the perspective image of another model.

The color correction table creation unit 33 obtains the hue of the different model image based on the plurality of different model images acquired via the acquisition unit 31 and the main model learning image stored in the storage unit 34. A "different model-> main model color correction table" for color correction to the hue of the learning image is created and stored in the storage unit 34. Further, the color correction table creation unit 33 determines the hue of the main model image based on the plurality of main model images acquired via the acquisition unit 31 and the different model learning image stored in the storage unit 34. A "main model-> another model color correction table" for color correction to the hue of another model learning image is created and stored in the storage unit 34.

In the storage unit 34, in addition to the above-mentioned main model learning image, another model learning image, another model → main model color correction table and main model → another model color correction table, the perspective images of the main model and another model are learned. , An "image learning program" for creating a main model learning image and another model learning image is stored. Further, the storage unit 34 has a determination operation of the contents (object) and the like shown by the perspective image of the main model whose color tone has been corrected, and a determination operation of the contents and the like shown by the perspective image of another model whose color tone has been corrected. An "image determination program" for controlling the image is stored. Further, the storage unit 34 stores a "color correction table creation program" for creating a different model-> main model color correction table and a main model-> different model color correction table.

The color correction unit 35 corrects the perspective image of another model to the perspective image of the hue of the main model based on the different model → main model color correction table. Further, the color correction unit 35 corrects the perspective image of the main model to the perspective image of the hue of the other model based on the main model → another model color correction table.

The determination unit 36, which is an example of the analysis unit, determines the contents of another model image color-corrected to the hue of the main model based on the main model learning image, and supplies the determination image to the monitor device 14. Further, the determination unit 36 determines the contents of the main model image color-corrected to the hue of another model based on the learning image of the other model, and supplies the determination image to the monitor device 14.

Note that some of the parts 31 to 36 shown in FIG. 3 may be realized by software.

(Software configuration of color correction table creation unit)
FIG. 4 is a functional block diagram of each function realized by software when the color correction table creation unit 33 executes the color correction table creation program stored in the storage unit 34. As shown in FIG. 4, the color correction table creation unit 33 executes the color correction table creation program to form the HLS conversion unit 41, the H histogram creation unit 42, the fixed width division unit 43, and the SL histogram creation unit 44. Realize each function. Further, the color correction table creation unit 33 realizes each function of the ranking unit 45, the same ratio rank detection unit 46, the conversion table forming unit 47, and the RGB conversion unit 48 by executing the color correction table creation program. The detailed operation of each part 41 to 48 will be described later.

(Judgment operation of another model image)
Next, the hue of the perspective image (another model image) formed by another model is corrected to the hue of the perspective image of the main model, and the different model is based on the corrected different model image and the main model learning image. A determination operation for determining the contents of the luggage shown in the image will be described.

FIG. 5 is a diagram showing an outline of a process of creating a color correction table for color correction of another model image. As shown in FIG. 5A, the object inspection device 1 as the main model learns a plurality of captured fluoroscopic images and the perspective image for the main model object inspection device 1 supplied from the outside and stores the storage unit. It is stored in 34 (main model learning image). Based on this main model learning image, the color correction table creation unit 33 of the object inspection device 1 which is the main model has hue (Hue), brightness (Lightness), and saturation (Saturation) as shown in FIG. 5 (c). ) HLS histograms are created respectively.

Further, the color correction table creation unit 33 is shown in FIG. 5 (d) based on an unlearned image (an unlearned image of another model) from the object inspection device 100 of another model as shown in FIG. 5 (b). As described above, HLS histograms of hue, brightness, and saturation are created respectively.

Next, the color correction table creation unit 33 has an appearance rate of each pixel value shown in the HLS histogram created from the main model learning image and an appearance rate of each pixel value shown in the HLS histogram created from the unlearned image of another model. Compare with. The fact that the appearance rate is the same between the HLS histogram of the main model learning image and the HLS histogram of the unlearned image of another model means that the pixel values are the same. Therefore, the color correction table creation unit 33 sets the pixel value of the HLS histogram of the unlearned image of another model, which has the same appearance rate as the appearance rate of the HLS histogram of the main model learning image, to the pixels of the main model learning image of the appearance rate. Create color correction information to correct to the value. The color correction table creation unit 33 creates a color correction table (separate model → main model color correction table) in which such color correction information is stored for each appearance rate and for each HLS histogram (FIG. 5 (e)). ).

By creating such a different model → main model color correction table, it is possible to determine the contents and the like based on the unlearned image of the different model in the object inspection device 1 which is the main model. FIG. 6 is a diagram showing an outline of this determination operation. As shown in FIG. 6A, when the fluoroscopic image (unlearned image of another model) captured by the object inspection device 100 of another model is supplied to the object inspection device 1 of the main model, the object inspection of the main model is performed. The color correction unit 35 of the device 1 refers to the color correction table of another model → main model (FIG. 6B), and corrects the hue of the perspective image of the other model to the hue for the object inspection device 1 of the main model. (Fig. 6 (c)).

The determination unit 36 of the object inspection device 1 of the main model determines the contents of the object shown in the perspective image of another model based on the perspective image of another model corrected to the hue for the object inspection device 1 of the main model. Judgment (FIG. 6 (d), FIG. 6 (e)). As a result, the object inspection device 1 of the main model can accurately determine the contents and the like based on the fluoroscopic image of another model.

(Details of color correction table formation operation)
The flowchart of FIG. 7 shows the flow of the above-mentioned color correction table creation operation. The color correction table creation unit 33 executes the color correction table creation program stored in the storage unit 34, and based on each function shown in FIG. 4 (HLS conversion unit 41 to RGB conversion unit 48), FIG. 7 Each process is executed from step S1 of the flowchart of.

First, in step S1, the HLS conversion unit 41 shown in FIG. 4 converts the RGB values of the plurality of master model learned images of the object inspection device 1 of the main model stored in the storage unit 34 into HLS values, and H The histogram creation unit 42 creates an H (hue) histogram as shown in FIG. 5 (c). That is, each pixel value of the master model trained image is a value in the RGB color space of red (R), green (G), and blue (B). Therefore, the HLS conversion unit 41 sets each pixel value, which is the value of the RGB color space of the image learned by the main model, into the value of the HLS color space of hue, brightness, and saturation. (HLS conversion process). The H-histogram creation unit 42 forms a hue (Hue) histogram among the HLS-converted HLS values.

FIG. 8 is a diagram showing an arithmetic expression used when converting an RGB value into an HLS value. As shown in FIG. 8, the HLS conversion unit 41 sets each maximum value (max) of RGB to "Vmax", sets each minimum value (min) of RGB to "Vmin", and sets "(Vmax + Vmin) / 2". The brightness L is calculated by performing the calculation.

Further, when the calculated brightness L value is less than 0.5 (L <0.5), the HLS conversion unit 41 calculates the saturation S by performing the calculation of "(Vmax-Vmin) / (Vmax + Vmin)". calculate. On the other hand, when the calculated brightness L value is 0.5 or more (L ≧ 0.5), the HLS conversion unit 41 performs the calculation of “(Vmax-Vmin) / (2- (Vmax + Vmin))”. By doing so, the saturation S is calculated.

Further, when the value of R is Vmax (Vmax = R), the HLS conversion unit 41 calculates the hue H by performing the calculation of "(60 (GB)) / (Vmax-Vmin)". .. Further, when the value of G is Vmax (Vmax = G), the HLS conversion unit 41 calculates the hue H by performing the calculation of "(120 + 60 (BR)) / (Vmax-Vmin)". .. Further, when the value of B is Vmax (Vmax = B), the HLS conversion unit 41 calculates the hue H by performing the calculation of "(240 + 60 (RG)) / (Vmax-Vmin)". ..

The RGB color space image of the main model trained image is converted into HSV color space values of hue (Hue), saturation (Saturation), and lightness (Value) to form a hue (Hue) histogram. May be good (HSV conversion).

When performing HSV conversion, as shown in FIG. 9, each maximum value (max) of RGB is set to brightness V. Further, when the value of V is not "0", the saturation S is calculated by performing the calculation of "(V-min (RGB)) / V". When the value of V is "0", the saturation S is also set to "0".

Further, when the value of R is V (V = R), the hue H is calculated by the calculation of "(60 (GB)) / (V-min (R, G, B))". When the value of G is V (V = G), the hue H is calculated by the calculation of "(120 + 60 (BR)) / (V-min (R, G, B))". When the value of B is V (V = B), the hue H is calculated by the calculation of "(240 + 60 (RG)) / (V-min (R, G, B))".

Next, when the H histogram creating unit 42 creates a histogram of the hue H, in step S2, the fixed width dividing unit 43 uses the histogram of the hue H, for example, 0 degrees to 29 degrees, 30 degrees to 30 degrees. 59 degrees, 60 degrees to 89 degrees ... Divide into 12 every 30 degrees of 330 degrees to 359 degrees. Thereby, the histogram of hue H can be divided into the same hue range such as red, red-yellow, yellow ... purple, magenta, and the like. Note that 12 divisions are an example, and the histogram of hue H may be divided into 10 divisions every 36 degrees.

The SL histogram creation unit 44 creates histograms of saturation S and brightness L for each of the divided colors. As a result, a histogram of the hue H of the image trained by the main model and a histogram of the saturation S and the brightness L for each fixed width (every 30 degrees) of the hue H are created.

Next, in step S3, the HLS conversion unit 41 shown in FIG. 4 converts the RGB values of a plurality of unlearned images of another model of the object inspection device of another model into HLS values, and the H histogram creation unit 42 converts the RGB values into HLS values. A histogram of H (hue) as shown in (d) is created in the same manner as described above.

When the H histogram creation unit 42 creates a histogram of the hue H of the unlearned image of another model, in step S4, the fixed width dividing unit 43 uses the histogram of the hue H of the unlearned image of another model, for example, on the hue circle. It is divided into fixed widths of 0 degrees to 29 degrees, 30 degrees to 59 degrees, 60 degrees to 89 degrees, and so on. The SL histogram creation unit 44 creates a histogram of the saturation S and the brightness L of the unlearned image of another model for each fixed width of the divided hue H.

As a result, a histogram of the hue H of the unlearned image of another model and a histogram of the saturation S and the brightness L for each fixed width of each hue H are created.

Next, in step S5, the ranking unit 45 creates a histogram of the hue H, saturation S, and brightness L of the main model learning image, and a histogram of the hue H, saturation S, and brightness L of the unlearned image of another model. The appearance rate for each appearing pixel value is ranked as shown in FIG. Further, in step S6, the same rate rank detection unit 46 detects the pixel value of the same rate rank among the ranked pixel values. Then, the conversion table forming unit 47 sets the hue H, saturation S, and brightness L of the unlearned image of another model to the hue H and color of the main model learning image based on the ranked pixel values. A color correction table (conversion table) for correcting each pixel value of hue H, saturation S, and brightness L of another model image corresponding to each pixel value of degree S and brightness L is formed.

More specifically, using the example of FIG. 10, in the case of the example of FIG. 10, there are two pixel values of 4 in the histogram of the hue H of the main model learning image (two are the same rate first place). An example is shown in which 5 pixel values of 5 exist (5 are in the same rate and 3rd place), and 6 pixel values of 6 are present (6 are in the same rate and 8th place).

Further, in the case of the example of FIG. 10, in the histogram of the hue H of the unlearned image of another model, there are two pixel values of 1 (two are the same ratio and the first place), and four pixel values of 2 are present (2). An example is shown in which four have the same rate of 3rd place, 4 have 5 pixel values (4 have the same rate of 7th place), and 3 have 6 pixel values (3 have the same rate of 11th place).

The conversion table forming unit 47 first forms a color correction table of H for converting the pixel values of each rank of the hue H of the unlearned image of another model into the pixel values of the rank corresponding to the learning image of the main model. That is, in the case of the example of FIG. 10, the pixel value of the pixel whose appearance rate is the first in the histogram of the hue H of the main model learning image is “4”, and the histogram of the hue H of the unlearned image of another model is used. The pixel value of the pixel having the highest appearance rate is "1". Therefore, the conversion table forming unit 47 has a pixel value of “1”, which has the highest appearance rate in the unlearned image of another model, and “4”, which has the highest appearance rate in the main model learning image. The color correction information to be converted into the pixel value of is stored in the color correction table (pixel value 1 → pixel value 4).

The pixel value of "0" in the unlearned image of another model is also converted into the pixel value of "0" in the learning image of the main model (0 → 0). Further, although it is an example, the color correction information stored in the color correction table is stored as 8-bit information.

Similarly, in the case of the example of FIG. 10, the pixel value of the pixel whose appearance rate is the third in the histogram of the hue H of the main model learning image is “5”, and the histogram of the hue H of the unlearned image of another model. The pixel values of the pixels having an appearance rate of 3rd to 7th are "2" to "5". Therefore, the conversion table forming unit 47 sets the pixel values of “2” to “5”, which have the third highest appearance rate in the histogram of the hue H of the unlearned image of another model, as the hue H of the main model learning image. The color correction information to be converted into the pixel value of "5" whose appearance rate is the third place in the histogram is stored in the color correction table (pixel value 2 → pixel value 5, pixel value 3 → pixel value 5, pixel value 4). → Pixel value 5, Pixel value 5 → Pixel value 5).

Similarly, in the case of the example of FIG. 10, the pixel value of the pixel whose appearance rate is 8th in the histogram of the hue H of the main model learning image is “6”, and the histogram of the hue H of the unlearned image of another model. The pixel value of the pixel whose appearance rate is 11th is "6". Therefore, the conversion table forming unit 47 uses the pixel value of “6”, which has an appearance rate of 11th in the hue H histogram of the unlearned image of another model, as the appearance rate in the hue H histogram of the main model learning image. The color correction information to be converted into the pixel value of "6" in which is the 8th place is stored in the color correction table (pixel value 6 → pixel value 6).

Next, as shown in FIG. 11, the conversion table forming unit 47 provides a color correction table having a brightness L corresponding to a hue H of 0 degrees to 29 degrees, 30 degrees to 59 degrees, 60 degrees to 89 degrees, and so on. , Similar to the above-mentioned hue H color correction table, it is created based on the histogram of the luminance L of the unlearned image of another model and the histogram of the luminance L of the main model learning image. The range of 0 to 29 degrees, the range of 30 degrees to 59 degrees, the range of 60 degrees to 89 degrees, and the like of hue H are substantially the same range of hues. Therefore, for each of the same colors, a color correction table for converting the brightness L of the unlearned image of another model into the brightness L of the learning image of the main model is created.

Further, as shown in FIG. 11, the conversion table forming unit 47 provides a color correction table having a saturation S corresponding to a hue H of 0 degrees to 29 degrees, 30 degrees to 59 degrees, 60 degrees to 89 degrees, and so on. , The color correction table of hue H described above is created based on the histogram of the saturation S of the unlearned image of another model and the histogram of the saturation S of the main model trained image. The range of 0 to 29 degrees, the range of 30 degrees to 59 degrees, the range of 60 degrees to 89 degrees, and the like of hue H are substantially the same range of hues. Therefore, for each of the same colors, a color correction table for converting the saturation S of the unlearned image of another model into the saturation S of the learning image of the main model is created.

(Adjustment operation when forming a color correction table)
Here, if a color correction table having hue H, brightness L, and saturation S is formed from a histogram in which pixel values are partially concentrated, an accurate color correction table cannot be obtained.

That is, to explain using the example of FIG. 12, in the histogram of the main model learning image, the appearance rate of the pixel of the pixel value of "0" is the first place, so that the appearance rate is 1 in the histogram of the unlearned image of another model. The six pixels with the pixel value of "1", which is the histogram, are converted into the pixels with the pixel value of "0" by the color correction table.

Similarly, in the example of FIG. 12, in the histogram of the main model learning image, the appearance rate of the pixel with the pixel value of "5" is 7th, so that the appearance rate is 7th and 9th in the histogram of the unlearned image of another model. Seven pixels with pixel values of "3" and "5" are converted into pixels with pixel values of "5" by the color correction table. As a result, by converting the histogram of the unlearned image of another model with the color correction table, there is an inconvenience that the histogram becomes different from the histogram of the learning image of the main model.

Therefore, as shown in FIG. 13 (a), the conversion table forming unit 47 displays the pixel values concentrated in a predetermined number or more in the histogram of the unlearned image of another model on the left and right as shown in FIG. 13 (b). Disperse to the pixel values of (= superimpose noise).

That is, in the example of FIG. 13A, there are 6 pixel values of "1" and 5 pixel values of "5", each of which is a predetermined number or more. Therefore, as shown in FIG. 13B, the conversion table forming unit 47 sets one of the six pixels having a pixel value of “1” as a pixel value of “0” and further sets one to “0”. Disperse to the pixel values of "3". Similarly, as shown in FIG. 13B, the conversion table forming unit 47 sets one of the five pixels having a pixel value of “5” as the pixel value of “4” and further sets one as “4”. It is distributed to the pixel values of "6".

Before noise superimposition (before dispersion), the color correction table based on the histogram of the main model learning image shown in FIG. 13 (c) caused the inconvenience that the same pixel values were concentrated as shown in FIG. 13 (d). ..

However, by superimposing noise to form the color correction table, the histogram of the image of another model can be approximated to the histogram of the learning image of the main model as shown in FIG. 13 (e).

When the color correction table of hue H, luminance L, and saturation S is formed in this way, in step S7 of the flowchart of FIG. 7, the RGB conversion unit 48 shown in FIG. 4 performs the hue H, luminance L, and saturation S. The HLS value of the color correction table of No. 1 is converted into an RGB value and stored in each color correction table again. As a result, a color correction table for R, a color correction table for G, and a color correction table for B for correcting the color of another model image to the hue of the main model image are formed, and are shown in the flowchart of FIG. The formation operation of the color correction table is completed. The color correction table for each color is stored in the storage unit 34 shown in FIG. 3 as “another model → main model color correction table”.

(Formation operation of color correction table when HSV value is used)
The above-mentioned explanation of the formation operation of the color correction table is an explanation when each pixel value of hue H, luminance L, and saturation S is used. As described above, instead of the hue H, the brightness L, and the saturation S, each pixel value of the hue H, the saturation S, and the brightness (brightness) V may be used.

First, FIG. 14 shows the characteristics of the hue H and the luminance value V in the HSV space. As shown in FIG. 14, the higher the atomic number of the substance constituting the object, the higher the hue H value, and the thicker the object thickness, the higher the brightness value V. However, due to deterioration noise of parts or compression noise of JPEG (Joint Photographic Experts Group) images, each value of hue H and brightness value V increases monotonically (the order of appearance of hues appearing between models is the same). In many cases, an object that does not transmit X-rays has the same value as an organic substance.

That is, in the object inspection system of the embodiment, it is assumed that each pixel value of HSV increases monotonically, but values close to white, black, and gray are assumed to increase monotonically because the value of hue H is easily changed. Becomes difficult.

Specifically, the value of hue H fluctuates greatly as follows, only when noise is superimposed on black (0, 0, 0) by "1".

RGB = (1,0,0) → H = 0 (red)
RGB = (0,1,0) → H = 120 (green)
RGB = (0,0,1) → H = 240 (blue)

Therefore, when the HSV value is used, pixels having a brightness value V of 50 or less are once converted into purple (hue H = 300 degrees) pixels, and the converted pixels are turned orange after the color correction table is created. It is converted and returned to bring the pixel value closer to black.

Specifically, when the HSV value is used, the color correction table creation unit 33 first superimposes noise on all the RGB images of the main model learning image and converts them into image information in the HSV space. Then, the pixels having a luminance value of 50 or less are converted into H = 150 (purple), and histograms of hue H, saturation S, and luminance value V are created in the same manner as described above. Further, the conversion table forming unit 47 creates a ranking of the appearance rate of each pixel based on the histogram of the hue H, the saturation S, and the luminance value V.

Next, the conversion table forming unit 47 superimposes noise on all the images of the unlearned images of another model, converts them into the HSV space, and converts the pixels having a brightness value of 50 or less into H = 150 (purple). Then, in the same manner as described above, histograms of hue H, saturation S, and luminance value V are created. Further, the conversion table forming unit 47 creates a ranking of the appearance rate of each pixel based on the histogram of the hue H, the saturation S, and the luminance value V.

Next, the conversion table forming unit 47 creates a color correction table for converting the histogram of the hue H of the unlearned image of another model into the histogram of the hue H of the main model learning image based on the ranking, as described above. Form. Further, the conversion table forming unit 47 divides the hue H into fixed widths of, for example, every 30 degrees as described above, and converts the histogram of the saturation S of the unlearned image of another model into the histogram of the saturation S of the main model learning image. A color correction table for converting to a histogram of brightness value V of an unlearned image of another model and a color correction table for converting a histogram of brightness value V of a learning image of another model are formed for each fixed width. ..

Next, the conversion table forming unit 47 approximates the pixel value to black by converting the pixel converted to purple into orange (the brightness value is reduced to, for example, 1/3). As a result, it is possible to prevent a large fluctuation in the hue H value due to superimposition of noise with respect to a value close to white, black, and gray, and it is possible to maintain a monotonous increase in each pixel value of HSV.

When the unlearned image of another model is edge-enhanced, the conversion table forming unit 47 performs the blurring process.

(Judgment operation)
Next, the flowchart of FIG. 15 shows the flow of the determination operation in the object inspection device 1 which is the main model. The operation of each unit shown in the flowchart of FIG. 15 is an operation based on the image determination program stored in the storage unit 34. That is, first, the acquisition unit 31 shown in FIG. 3 determines whether or not the acquired image (see FIG. 6A) is a different model image (step S11). When the acquired image is the main model image (step S11: No), the process proceeds to step S13. If the acquired image is a different model image (step S11: Yes), the process proceeds to step S12.

In step S12, the color correction unit 35 converts each pixel value of the acquired image of another model into each pixel value stored in the storage unit 34 from another model to the main model conversion table (FIG. 6). (B), see FIG. 6 (c)). As a result, the hue of the image of another model can be converted into the hue of the image of the main model.

Next, the determination unit 36 was acquired in step S11 or the contents of an object shown in another model image whose hue has been converted based on the main model learning image stored in the storage unit 34. The contents of the object shown in the main model image are determined (see FIGS. 6 (d) and 6 (e)). Then, the determination unit 36 transmits the determination image resulting from this determination result to the monitoring device 14 of the object inspection device 1 shown in FIGS. 1 and 2.

The example shown in FIG. 6E is an example of a determination image when a knife, which is a dangerous substance, is included in the baggage. Even when making a judgment based on a different model image, the color of the different model image is converted to the color of the main model image by the different model → main model conversion table, so the judgment unit 36 makes a special judgment for the different model image. Accurate judgment can be made only by performing the normal judgment operation for the main model without performing the operation. The inspector determines whether or not there is an open cover inspection based on such a judgment image, and performs an open cover inspection of the luggage as necessary.

(Forming operation of the color correction table of the main model image)
The above description is an example of forming a color correction table that corrects the hue of another model image to the hue of the main model image, but the color correction table that corrects the hue of the main model image to the hue of the other model image. Can also be formed. FIG. 16 is a flowchart showing the flow of the formation operation of the color correction table for color correction of the main model image. The color correction table creation unit 33 executes the color correction table creation program stored in the storage unit 34, and based on each function shown in FIG. 4 (HLS conversion unit 41 to RGB conversion unit 48), FIG. Each process is executed from step S21 of the flowchart of.

First, in step S21, the HLS conversion unit 41 shown in FIG. 4 converts the RGB values of a plurality of different model images of the object inspection device of another model stored in the storage unit 34 into HLS values, and the H histogram creation unit. 42 creates a histogram of H (hue) shown in FIG. 5 (c).

Next, when the H histogram creating unit 42 creates a histogram of the hue H, in step S22, the fixed width dividing unit 43 uses the histogram of the hue H, for example, 0 degrees to 29 degrees, 30 degrees to 30 degrees. 59 degrees, 60 degrees to 89 degrees ... Divide into 12 every 30 degrees of 330 degrees to 359 degrees.

The SL histogram creation unit 44 creates histograms of saturation S and brightness L for each of the divided colors. As a result, a histogram of the hue H of another model image and a histogram of the saturation S and the brightness L for each fixed width (every 30 degrees) of the hue H are created.

Next, in step S23, the HLS conversion unit 41 shown in FIG. 4 converts the RGB values of the plurality of main model images of the object inspection device 1 of the main model into HLS values, and the H histogram creation unit 42 displays FIG. A histogram of H (hue) as shown in d) is created in the same manner as described above.

When the H-histogram creation unit 42 creates a histogram of the hue H of the main model image, in step S24, the fixed width dividing unit 43 creates a histogram of the hue H of the main model image, for example, 0 degrees to 29 degrees, 30 degrees. It is divided into fixed widths of degrees to 59 degrees, 60 degrees to 89 degrees, and so on. The SL histogram creation unit 44 creates a histogram of the saturation S and the brightness L of the main model image for each fixed width of the divided hue H.

As a result, a histogram of the hue H of the unlearned image of the main model and a histogram of the saturation S and the brightness L for each fixed width of each hue H are created.

Next, in step S25, the ranking unit 45 appears in the histogram of the hue H, the saturation S and the brightness L of the main model image, and the histogram of the hue H, the saturation S and the brightness L of the other model image. The appearance rate for each pixel value is ranked as shown in FIG. Further, in step S26, the same rate rank detection unit 46 detects the pixel value of the same rate rank among the ranked pixel values. Then, the conversion table forming unit 47 sets each pixel value of the hue H, saturation S, and brightness L of the main model image to the hue H, saturation S, and the hue S of another model image based on the ranked pixel values. A color correction table (conversion table) for correcting each pixel value of the hue H, saturation S, and brightness L of the main model image corresponding to each pixel value of the brightness L is formed.

Next, as shown in FIG. 11, the conversion table forming unit 47 is a color correction table having a brightness L corresponding to a hue H of 0 degrees to 29 degrees, 30 degrees to 59 degrees, 60 degrees to 89 degrees, and so on. Is created based on the histogram of the luminance L of the main model image and the histogram of the luminance L of the other model image in the same manner as the above-mentioned hue H color correction table. The range of 0 to 29 degrees, the range of 30 degrees to 59 degrees, the range of 60 degrees to 89 degrees, and the like of hue H are substantially the same range of hues. Therefore, for each of the same colors, a color correction table for converting the brightness L of the main model image to the brightness L of the different model image is formed.

Further, as shown in FIG. 11, the conversion table forming unit 47 is a color correction table having saturation S corresponding to hues H of 0 degrees to 29 degrees, 30 degrees to 59 degrees, 60 degrees to 89 degrees, and so on. Is created based on the histogram of the saturation S of the main model image and the histogram of the saturation S of the other model image in the same manner as the above-mentioned hue H color correction table. The range of 0 to 29 degrees, the range of 30 degrees to 59 degrees, the range of 60 degrees to 89 degrees, and the like of hue H are substantially the same range of hues. Therefore, for each of the same colors, a color correction table for converting the saturation S of the main model image to the saturation S of another model image is created.

Next, when the color correction table of hue H, luminance L, and saturation S is formed in this way, in step S27 of the flowchart of FIG. 16, the RGB conversion unit 48 shown in FIG. The HLS value of the color correction table of saturation S is converted into an RGB value and stored in each color correction table again. As a result, a color correction table for R, a color correction table for G, and a color correction table for B for correcting the color tone of the main model image to the color tone of another model image are formed, respectively, and the flowchart of FIG. 16 shows. The formation operation of the color correction table shown is completed. The color correction table for each color is stored in the storage unit 34 shown in FIG. 3 as a “main model → another model color correction table”.

(Judgment operation)
Next, the flow chart of FIG. 17 shows the flow of the determination operation in which the object inspection device 1 as the main model converts the hue of the main model image into the hue of the image of another model to determine the contents of the object. show. The operation of each unit shown in the flowchart of FIG. 17 is an operation based on the image determination program stored in the storage unit 34. That is, first, the acquisition unit 31 shown in FIG. 3 determines whether or not the acquired image is the main model image (step S31). When the acquired image is the main model image (step S31: Yes), the process proceeds to step S32. If the acquired image is a different model image (step S31: No), the process proceeds to step S33.

In step S32, the color correction unit 35 converts each pixel value of the acquired main type image into each pixel value stored in the storage unit 34 from the main model to another model conversion table. As a result, the hue of the main model image can be converted into the hue of the image of another model.

Next, the determination unit 36 determines the contents of the object shown in the main model image that has been converted into the hue of the different model image based on the different model learning image stored in the storage unit 34. Alternatively, the determination unit 36 determines the contents of the object shown in the image of another model acquired in step S31 based on the "learning image of another model" stored in the storage unit 34. Then, the determination unit 36 transmits the determination image resulting from this determination result to the monitoring device 14 of the object inspection device 1 shown in FIGS. 1 and 2. The operation of forming the "different model learning image" will be described later with reference to FIGS. 20 and 21.

In this way, the color tone of the main model image can be converted to the color tone of the different model image by the main model → different model conversion table, and the determination can be made based on the different model learning image. The inspector determines whether or not there is an open cover inspection based on such a judgment image, and performs an open cover inspection of the baggage as necessary.

(Learning operation of main model learning image)
Next, the determination unit 36 will explain the learning operation of the main model learning image used for determining the contents of the object and the like. FIG. 18 is a schematic diagram of the learning operation of the main model learning image. Main model learning Image learning can be learned (memorized) in two ways. First, every time an image is taken by the object inspection device 1 which is the main model, all the captured fluoroscopic images or the selected fluoroscopic images are stored in the storage unit 34. As a result, each time the object inspection device 1 takes an image, the fluoroscopic image captured by the main model object inspection device 1 is stored in the storage unit 34 as the main model learning image.

Further, when a fluoroscopic image (another model image) is supplied from an object inspection device of another model, as shown in FIG. 18, the color tone of the other model image is changed to the main model by the above-mentioned different model → main model conversion table. Convert to image tint. The image learning unit 32 stores the image of another model whose color has been converted in the storage unit 34 as the main model learning image. As described above, by forming the conversion table from another model to the main model, the fluoroscopic images of both the object inspection device 1 as the main model and the object inspection device as the other model are learned (memory) as the main model learning image. can do.

FIG. 19 is a schematic diagram showing how the contents and the like are determined based on the main model learning image learned in this way. When the fluoroscopic image is captured by the object inspection device 1 as the main model, the determination unit 36 makes a determination based on the main model learning image stored in the storage unit 34 as shown in FIG. 19A. .. Further, when a fluoroscopic image (another model image) from an object inspection device of another model is supplied, the determination unit 36 uses a different model → main model conversion table to obtain a different model, as shown in FIG. 19 (b). After converting the hue of the image into the hue of the main model image, the determination is performed based on the main model learning image stored in the storage unit 34. As described above, since it has a conversion table from another model to the main model, the fluoroscopic images of both the object inspection device 1 as the main model and the object inspection device as the other model are determined based on the main model learning image. can do.

(Learning operation of another model learning image)
Next, the determination unit 36 will explain the learning operation of the learning image of another model used for determining the contents of the object and the like. FIG. 20 is a schematic diagram of a learning operation of another model learning image. In the learning of the different model learning image, the color tone of the captured image (main model image) captured by the main model object inspection device 1 is changed from the main model to the different model conversion table described above to obtain the different model of the object inspection device. Convert to the shade of another model image. The image learning unit 32 stores the main model image whose hue has been converted in the storage unit 34 as another model learning image. As a result, every time an image is taken by the object inspection device 1, another model image formed by the main model → another model conversion table is stored in the storage unit 34 as another model learning image.

FIG. 21 is a schematic diagram showing how the contents and the like are determined based on the different model learning image learned in this way. When a different model image of the different model object inspection device is supplied to the main model object inspection device 1, the determination unit 36 uses the different model learning image stored in the storage unit 34 as shown in FIG. 21. Judgment is made based on. The different model learning image is an image converted into a hue for another model image by the main model → different model conversion table. Therefore, it is possible to determine the contents and the like without correcting the hue of the image of another model. Therefore, high-speed determination can be made possible.

(Effect of embodiment)
As is clear from the above description, the object inspection system of the embodiment uses the first image information of the first image format (RGB format) as the second image including the hue H, the saturation S, and the brightness L. It has a format conversion unit (HLS conversion unit 41) that converts the second image information of the format (HLS format).

Further, it has a first histogram creating unit (H histogram creating unit 42) that creates a histogram of hue H, and a fixed width dividing unit 43 that divides hue information at predetermined angles (for each fixed width) on the hue circle. .. Further, it has a second histogram creating unit (SL histogram creating unit) 44 that creates a histogram of saturation S and brightness L for each divided angle.

Further, the appearance rate of each pixel value shown by the histogram of hue H, the histogram of saturation S and the histogram of brightness L corresponding to the first hue (the hue of the main model image) has a second hue different from that of the first hue. Colors of hue H, saturation S, and brightness L for matching the appearance rates of each pixel value shown in the histogram of hue H, the histogram of saturation S, and the histogram of brightness L corresponding to the hue (the hue of another model image). It has a color correction information creating unit (ranking unit 45, synchronism ranking detecting unit 46, and conversion table forming unit 47) for creating correction information (color correction table).

Further, a conversion unit (RGB conversion unit 48) that converts each color correction information of hue H, saturation S, and brightness L into color correction information corresponding to the first image format (RGB format), and a first hue (RGB conversion unit 48). The first image information (another model image) of the first image format of the first image format having a second hue different from the hue of the main model image) is of the first hue based on each color correction information converted by the conversion unit. It has a color conversion unit (color correction unit 35) that converts the first image information (main model image).

In addition, a learning unit (image) that stores and stores the first image information (another model image) converted into the first hue as the first image information (main model learning image) for object determination in the storage unit. It has a learning unit 32).

As a result, it is possible to construct a learning model that can correct the difference in the coloring process between the manufacturers for the fluoroscopic image and obtain an accurate determination result of the contents and the like. Further, since the histograms of the color S and the brightness L are created for each hue of the same hue (for each of the above-mentioned fixed widths), it is possible to prevent the inconvenience that a non-existent color appears in the image of another model after the color conversion. Therefore, it is possible to construct a learning model capable of improving the accuracy of the determination result of the contents and the like.

Further, in the object inspection system of the embodiment, the first histogram creation unit (H histogram creation unit 42) and the second histogram creation unit (SL histogram creation unit 44) have the pixel values of the histograms of the respective pixel values. A part of the pixel values of the histograms in which the number of histograms is equal to or greater than a predetermined number is dispersed in the left and right histograms to create each histogram (see FIG. 13). This makes it possible to create a color correction table capable of accurate conversion.

Finally, the embodiments described above are presented as an example and are not intended to limit the scope of the invention. This novel embodiment can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. Further, the embodiment and the modification of the embodiment are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalent scope thereof.

1 Object inspection device 2 Analysis device 4 Luggage 14 Monitor device 16 Main body of object inspection device 16a Luggage inlet 16b Luggage outlet 21 X-ray source 22 Roller conveyor 23 X-ray detector 24 Image processing unit 31 Acquisition unit 32 Image learning unit 33 Color correction table creation unit 34 Storage unit 35 Color correction unit 36 Judgment unit 41 HLS conversion unit 42 H histogram creation unit 43 Fixed width division unit 44 SL histogram creation unit 45 Ranking unit 46 Equal rate rank detection unit 47 Conversion table formation unit 48 RGB Converter

Claims (4)

A format conversion unit that converts the first image information of the first image format into the second image information of the second image format including hue information, saturation information, and luminance information.
A histogram creation unit that divides the hue information for each predetermined angle on the hue circle and creates a histogram of the hue information, the saturation information, and the luminance information for each divided angle.
The hue information corresponding to the first hue, the saturation information, and the appearance rate of each pixel value shown by the histogram of the brightness information, and the hue information corresponding to the second hue different from the first hue. , A color correction information creation unit that creates color correction information of the hue information, the saturation information, and the brightness information for matching the appearance rates of each pixel value shown by the saturation information and the histogram of the brightness information. When,
A conversion unit that converts each of the hue information, the saturation information, and the color correction information of the brightness information into color correction information corresponding to the first image format.
The first image information of the first image format having a second hue different from the first hue is subjected to the first image information of the first hue based on the color correction information converted by the conversion unit. A color conversion unit that converts to the first image information,
Construction of a learning model characterized by having a learning unit that stores and stores the first image information converted into the first color as the first image information for object determination in a storage unit. Device. The histogram creating unit is characterized in that, among the histograms of each pixel value, a part of the pixel values of the histogram in which the number of pixel values is a predetermined number or more is dispersed in the left and right histograms to create each of the histograms. The device for constructing a learning model according to claim 1. A format conversion step in which the format conversion unit converts the first image information of the first image format into the second image information of the second image format including hue information, saturation information, and luminance information.
A histogram creation step in which the histogram creation unit divides the hue information at a predetermined angle on the hue circle and creates a histogram of the hue information, the saturation information, and the luminance information for each divided angle.
The color correction information creation unit has a second hue that is different from the first hue in the appearance rate of each pixel value shown by the histogram of the hue information, the saturation information, and the brightness information corresponding to the first hue. The hue information, the saturation information, and the color correction information of the brightness information for matching the appearance rate of each pixel value shown in the histogram of the hue information, the saturation information, and the brightness information corresponding to the hue. The color correction information creation step to be created and
A conversion step in which the conversion unit converts each of the hue information, the saturation information, and the color correction information of the luminance information into color correction information corresponding to the first image format.
The color conversion unit uses the first image information of the first image format having a second hue different from the first hue, based on the color correction information converted by the conversion unit. A color conversion step for converting one color to the first image information, and
A learning step in which the learning unit stores and stores the first image information converted into the first hue in the storage unit as the first image information for object determination.
A method of constructing a learning model characterized by having. Computer,
A format conversion unit that converts the first image information of the first image format into the second image information of the second image format including hue information, saturation information, and luminance information.
A histogram creation unit that divides the hue information for each predetermined angle on the hue circle and creates a histogram of the hue information, the saturation information, and the luminance information for each divided angle.
The hue information corresponding to the first hue, the saturation information, and the appearance rate of each pixel value shown by the histogram of the brightness information, and the hue information corresponding to the second hue different from the first hue. , A color correction information creation unit that creates color correction information of the hue information, the saturation information, and the brightness information for matching the appearance rates of each pixel value shown by the saturation information and the histogram of the brightness information. When,
A conversion unit that converts each of the hue information, the saturation information, and the color correction information of the brightness information into color correction information corresponding to the first image format.
The first image information of the first image format having a second hue different from the first hue is subjected to the first image information of the first hue based on the color correction information converted by the conversion unit. A color conversion unit that converts to the first image information,
To function as a learning unit that stores and stores the first image information converted into the first color as the first image information for object determination in the storage unit.
A learning model construction program featuring.

Images