WO2023084644A1

WO2023084644A1 - Information processing device, information processing method, and information processing program

Info

Publication number: WO2023084644A1
Application number: PCT/JP2021/041367
Authority: WO
Inventors: 啓太吉田; 徹平藤原; 高之天見
Original assignee: 三菱電機株式会社
Priority date: 2021-11-10
Filing date: 2021-11-10
Publication date: 2023-05-19
Also published as: JPWO2023084644A1; JP7066082B1

Abstract

An information processing device according to the present invention is provided with a data acquisition unit, a brightness correction unit, an image superimposing unit, a deformation detection unit, and a parameter determination unit. The data acquisition unit acquires data of a plurality of images acquired by capturing images of a target object by means of a plurality of image capturing devices. The brightness correction unit corrects brightness of each of the plurality of images. The image superimposing unit generates a superimposed image, which is an image acquired by superimposing the plurality of images on which the brightness correction by the brightness correction unit has been performed. The deformation detection unit detects a deformed part of the target object by using a learning model for detecting a deformed part of the target object from the superimposed image. The parameter determination unit determines a parameter to be used for the brightness correction on the basis of accuracy of the deformed part detection by the deformation detection unit.

Description

Information processing device, information processing method, and information processing program

The present disclosure relates to an information processing device, an information processing method, and an information processing program for determining parameters used for correcting the brightness of multiple images obtained by imaging an object with multiple imaging devices.

Conventionally, there is known a measurement vehicle in which a plurality of imaging devices with different imaging directions are installed to inspect an object such as a tunnel, and the object is imaged by these imaging devices while the vehicle is traveling. In such a measurement vehicle, imaging conditions for an imaging target, such as focus or iris, are switched for each imaging device.

Japanese Patent Laid-Open No. 2002-200000 discloses a technology for automatically adjusting imaging conditions, which is installed in a vehicle based on driving environment information including information such as the presence or absence of tunnels, the presence or absence of overpasses, road gradients, date and time, weather, and direction of the vehicle. Techniques for adjusting the exposure conditions of an imaging device have been proposed.

JP 2006-222844 A

However, when detecting a deformed portion of an object from a stitched image, which is an image obtained by photographing an object with a plurality of imaging devices and then stitching the images together after correcting the luminance, the modified portion of the object is detected by correcting the luminance. In some cases, the accuracy of detection of irregularities may decrease.

The present disclosure has been made in view of the above, and provides an information processing apparatus capable of appropriately determining parameters capable of correcting brightness while suppressing deterioration in detection accuracy of a deformed portion of an object. with the aim of obtaining

In order to solve the above-described problems and achieve the object, the information processing device of the present disclosure includes a data acquisition unit, a luminance correction unit, an image stitching unit, a deformation detection unit, and a parameter determination unit. . The data acquisition unit acquires data of a plurality of images obtained by imaging an object with a plurality of imaging devices. The luminance corrector corrects the luminance of each of the plurality of images. The image stitching unit generates a stitched image that is an image obtained by stitching together a plurality of images whose luminance has been corrected by the luminance correction unit. The deformation detection unit detects a deformation portion of the object using a learning model for detecting a deformation portion of the object from the combined images. The parameter determination unit determines a parameter used for luminance correction based on the accuracy of detection of the deformation location by the deformation detection unit.

According to the present disclosure, it is possible to appropriately determine a parameter capable of correcting luminance while suppressing deterioration in detection accuracy of a deformed portion of an object.

1 is a diagram showing an example of a configuration of a measurement processing system according to a first embodiment; FIG. FIG. 2 is a diagram showing an example of the configuration of a measurement device included in the measurement vehicle according to the first embodiment; FIG. FIG. 2 is a diagram showing an example of arrangement of a plurality of imaging devices in the measuring device according to the first embodiment; 1 is a diagram showing an example of a configuration of an information processing apparatus according to a first embodiment; FIG. FIG. 4 is a diagram showing an example of a captured image data table stored in the captured image data storage unit according to the first embodiment; FIG. FIG. 4 shows an example of a parameter candidate table stored in the parameter candidate storage unit according to the first embodiment; FIG. 4 is a diagram showing an example of a parameter table stored in the parameter storage unit according to the first embodiment; FIG. FIG. 4 is a diagram showing an example of parameter determination processing executed by the processing unit of the information processing apparatus according to the first embodiment; FIG. 4 is a diagram for explaining gamma correction executed by a luminance correction unit of the information processing apparatus according to the first embodiment; FIG. 4 is a diagram for explaining a local histogram equalization method executed by the luminance correction unit of the information processing apparatus according to the first embodiment; FIG. 4 is a diagram for explaining high-pass filtering performed by the luminance correction unit of the information processing apparatus according to the first embodiment; FIG. 4 is a diagram for explaining an example of captured image stitching processing by an image stitching unit of the information processing apparatus according to the first embodiment; FIG. 4 is a diagram for explaining a method of calculating a recall rate, a precision rate, and an F value by an evaluation unit of the information processing apparatus according to the first embodiment; FIG. 4 is a diagram showing an example of an inspection image generated by an inspection image generation unit of the information processing apparatus according to the first embodiment; 4 is a flowchart showing an example of processing by a processing unit of the information processing apparatus according to the first embodiment; 4 is a flowchart showing an example of parameter determination processing by the processing unit of the information processing apparatus according to the first embodiment; 3 is a flowchart showing an example of deformation detection accuracy evaluation processing by the processing unit of the information processing apparatus according to the first embodiment; 4 is a flowchart showing an example of manual mode processing by the processing unit of the information processing apparatus according to the first embodiment; 4 is a flowchart showing an example of automatic mode processing by the processing unit of the information processing apparatus according to the first embodiment; 3 is a flowchart showing an example of inspection image generation processing by the processing unit of the information processing apparatus according to the first embodiment; 1 is a diagram showing an example of a hardware configuration of an information processing apparatus according to a first embodiment; FIG.

The information processing device, information processing method, and information processing program according to the embodiment will be described in detail below with reference to the drawings.

Embodiment 1.
1 is a diagram illustrating an example of a configuration of a measurement processing system according to a first embodiment; FIG. As shown in FIG. 1 , the measurement processing system 100 according to the first embodiment includes an information processing device 1 and a measurement vehicle 2 . The measurement vehicle 2 includes a vehicle body 60 and a measurement device 70 mounted on the vehicle body 60 . The measurement device 70 repeatedly captures images of objects existing around the vehicle body 60 as objects to be measured while the vehicle body 60 is running.

In the example shown in FIG. 1, the objects to be measured by the measuring device 70 are the tunnel 4 and the guardrail 5, but are not limited to such examples. It may be an object or the like. In the example shown in FIG. 1, the measurement vehicle 2 is a vehicle that travels on a road and is an automobile that travels on roads, but may be a railroad vehicle that travels on rails.

The measuring device 70 generates captured image data including data of an image obtained by capturing an object. The measuring device 70 can transmit and receive data to and from the information processing device 1 via the network 3 , and transmits measurement data including captured image data to the information processing device 1 . The network 3 is, for example, a WAN (Wide Area Network) such as the Internet, but may be a LAN (Local Area Network) or other networks.

FIG. 2 is a diagram showing an example of the configuration of the measurement device provided in the measurement vehicle according to the first embodiment. As shown in FIG. 2 , the measurement device 70 included in the measurement vehicle 2 includes

imaging devices

71 a , 71 b , and 71 c , a position/attitude velocity detection unit 72 , a processing unit 73 , and a communication unit 74 .

The

imaging devices

71a, 71b, and 71c have imaging directions in directions orthogonal to the traveling direction of the vehicle body 60 and in different directions. FIG. 3 is a diagram illustrating an example of arrangement of a plurality of imaging devices in the measuring device according to the first embodiment; In the example shown in FIG. 3, the imaging direction of the imaging device 71a is the upper left direction, the imaging direction of the imaging device 71b is the upper direction, and the imaging direction of the imaging device 71c is the upper right direction. be.

In the example shown in FIG. 3, images of the inner wall surface of the tunnel 4 are captured by a plurality of

imaging devices

71a, 71b, and 71c. The

imaging devices

71a, 71b, and 71c are, for example, color line sensors, and repeatedly capture images of mutually different regions of the inner wall surface 4a that partially overlap each other while the vehicle body 60 is running. In the following description, the

imaging devices

71a, 71b, and 71c may be referred to as the imaging device 71 when they are not individually distinguished.

The imaging device 71 switches imaging conditions for the object by changing imaging parameter settings. The content of the setting change of the imaging parameter is set in advance according to the state of the imaging environment such as the state of the tunnel 4, the time of day, the weather, and the season. The setting may be changed automatically. The imaging parameters are, for example, at least one of focus, iris, gain control, dynamic range, and the like.

Note that the imaging device 71 may be a color area sensor or a monochrome sensor instead of the color line sensor. Also, the number of imaging devices 71 provided in the measuring device 70 is not limited to three, and may be two or less, or may be four or more, for example.

The position/attitude/speed detection unit 72 shown in FIG. 2 includes a GPS (Global Positioning System) receiver, an inertial sensor, and a speed sensor, and detects the position, attitude, and speed of the vehicle body 60 . The processing unit 73 controls the plurality of imaging devices 71 based on, for example, the position, orientation, and speed of the vehicle body 60, and causes the plurality of imaging devices 71 to repeatedly perform imaging of the target object.

In addition, the processing unit 73 connects, for each imaging device 71, a plurality of images repeatedly captured by each imaging device 71 in the traveling direction of the vehicle body 60 based on the position, posture, and speed of the vehicle body 60, for example. Captured image data for each imaging device 71, which is the captured image data, is generated.

The processing unit 73 outputs measurement data including captured image data for each imaging device 71 to the communication unit 74 . The communication unit 74 is wirelessly connected to the network 3 and transmits measurement data acquired from the processing unit 73 to the information processing device 1 via the network 3 .

The information processing device 1 generates a composite image by combining a plurality of captured images in a direction orthogonal to the traveling direction of the vehicle body 60 based on the measurement data acquired from the measurement vehicle 2, and generates the composite image. Based on, a deformed portion of the object is detected. A deformed portion is a portion where deformation occurs. For example, when the object is the tunnel 4, the deformation is cracking, peeling, staining, water leakage, or the like of the inner wall surface 4a of the tunnel 4.

The information processing device 1 generates an inspection image by performing a process of arranging information indicating the detected deformed portion on the composite image, and displays or prints the generated inspection image. Such an inspection image allows the user of the information processing apparatus 1, as an inspection operator, to easily grasp the deformed portion of the object and to paste the inspection image onto a form or the like.

A plurality of captured images obtained from the corresponding imaging devices 71 under mutually different imaging conditions may have a luminance difference. Visibility of the stitched image may deteriorate due to luminance unevenness in the stitched image. In addition, when the contrast of a plurality of captured images is poor or when the luminance is low, the visibility of the composite image may be deteriorated.

Therefore, the information processing apparatus 1 corrects the luminance of a plurality of captured images before generating a composite image, and generates a composite image by combining the plurality of images whose luminance has been corrected. As a result, the visibility of the stitched image in the inspection image can be improved compared to the stitched image obtained by stitching a plurality of images without correcting the brightness.

If the user arbitrarily sets a parameter P for correcting the brightness of a plurality of captured images to be stitched together, or automatically corrects the brightness from the histogram shape of each captured image to be stitched together. , there is a possibility that the detection accuracy for detecting a deformed portion from the composite image will be lower than in the case where luminance correction processing is not performed. For example, brightness correction may cause overexposure or underexposure in the deformed portion of the stitched image, making it impossible to detect the deformed portion.

Therefore, the information processing device 1 determines the parameter P for correcting the luminance based on the detection accuracy of the deformed portion from the stitched image, which is an image obtained by stitching together a plurality of captured images whose luminance is corrected. As a result, the information processing apparatus 1 appropriately determines the parameter P that can correct the brightness while suppressing deterioration in detection accuracy of the deformed portion of the object. The configuration of the information processing apparatus 1 will be specifically described below.

FIG. 4 is a diagram showing an example of the configuration of the information processing device according to the first embodiment. As shown in FIG. 4 , the information processing device 1 includes a communication section 10 , an input section 11 , a display section 12 , a storage section 13 and a processing section 14 .

The communication unit 10 is communicably connected to the network 3 by wire or wirelessly, and transmits and receives information to and from an external device such as a measuring device 70, a printer (not shown), or a terminal device via the network 3.

The input unit 11 includes, for example, a mouse and keyboard, but may also be a touch pad. The display unit 12 is, for example, a liquid crystal display, an organic EL (ElectroLuminescence) display, or a projector. In the example shown in FIG. 4, the input unit 11 and the display unit 12 are included in the information processing device 1. The input unit 11 and the display unit 12 are connected to the information processing device 1 as external devices of the information processing device 1. may be

The storage unit 13 includes a captured image data storage unit 20, a parameter candidate storage unit 21, and a parameter storage unit 22. The captured image data storage unit 20 stores a captured image data table including captured image data for each imaging device 71 included in the measurement data transmitted from the measuring device 70 .

FIG. 5 is a diagram showing an example of a captured image data table stored in a captured image data storage unit according to the first embodiment. The captured image data table shown in FIG. 5 includes “target object ID (IDentifier)”, “imaging device ID”, and “captured image data” for each captured image data.

"Object ID" is unique identification information for each object. “Imaging device ID” is unique identification information for each imaging device 71 . “Captured image data” is captured image data or information indicating the storage location of captured image data.

In the captured image data table shown in FIG. 5, the captured image data of the object with the object ID "M1" is captured by the

imaging devices

71a, 71b, and 71c with the imaging device IDs "DE1," "DE2," and "DE3." It is shown that the captured image data generated based on the images obtained are the captured image data IMD1, IMD2, and IMD3.

The parameter candidate storage unit 21 shown in FIG. 4 stores a parameter candidate table containing a plurality of parameter candidates that are candidates for the parameter P used in the brightness correction process. 6 is a diagram of an example of a parameter candidate table stored in a parameter candidate storage unit according to the first embodiment; FIG.

The parameter candidate table shown in FIG. 6 includes "object ID" and "parameter candidate". The "object ID" is the same as the "object ID" shown in FIG. The “parameter candidate” is information of a parameter candidate that is a candidate of the parameter P used for the luminance correction process.

In the parameter candidate table shown in FIG. 6, parameter candidates Pc1, Pc2, . n is an integer of 4 or more, for example. Further, parameter candidates Pc2 and the like are included as parameter candidates for the object with the object ID “M2”. In the following description, parameter candidates Pc1, Pc2, . Note that the parameter candidate table may include identification information unique to the type of object instead of or in addition to the object ID.

The parameter storage unit 22 shown in FIG. 4 stores a parameter table including parameters P used for luminance correction processing. 7 is a diagram of an example of a parameter table stored in a parameter storage unit according to the first embodiment; FIG.

The parameter table shown in FIG. 7 includes "object ID" and "parameter". The "object ID" is the same as the "object ID" shown in FIG. “Parameter” is information of a parameter P used for luminance correction processing. In the parameter table shown in FIG. 7, the parameter candidate Pc2 is set as the parameter P for the object with the object ID "M1", and the parameter candidate Pcn is set as the parameter P for the object with the object ID "M2". there is Note that the parameter table may include identification information unique to the type of object instead of or in addition to the object ID.

Returning to FIG. 4, the description of the information processing device 1 will be continued. The processing unit 14 shown in FIG. 4 performs parameter determination processing for determining a parameter P for luminance correction when a parameter determination request is received through an input operation to the input unit 11 by the user of the information processing apparatus 1 . 8 is a diagram illustrating an example of parameter determination processing performed by the processing unit of the information processing apparatus according to the first embodiment; FIG.

FIG. 8 shows an example in which captured images A, B, and C are pasted together as a plurality of captured images. The captured image A is, for example, a captured image represented by the above-described captured image data IMD1, the captured image B is, for example, a captured image represented by the above-described captured image data IMD2, and the captured image C is, for example, , and the captured image represented by the captured image data IMD3 described above.

The processing unit 14 performs luminance correction processing for correcting the luminance of each of the captured images A, B, and C based on the parameter candidates Pc1, Pc2, . . As a result, for example, the captured images A, B, and C corrected for brightness using the parameter candidate Pc1, the captured images A, B, and C corrected for brightness using the parameter candidate Pc2, and the brightness corrected using the parameter candidate Pcn. The picked-up images A, B, C, etc. are generated.

Next, the processing unit 14 performs a stitching process of generating a stitched image, which is an image obtained by stitching together the picked-up images A, B, and C whose brightness has been corrected, for each parameter candidate Pc. As a result, composite images IC _Pc1 , IC _Pc2 , . . . , IC _Pcn are generated.

A stitched image IC _Pc1 is an image obtained by stitching captured images A, B, and C corrected for brightness using the parameter candidate Pc1, and a stitched image IC _Pc2 is an image obtained by performing brightness correction using the parameter candidate Pc2. It is an image in which images A, B, and C are pasted together. A stitched image IC _Pcn is an image obtained by stitching together the captured images A, B, and C whose brightness has been corrected using the parameter candidate Pcn. In _the _following , when each of the composite images IC _Pc1 , IC _Pc2 , .

Next, the processing unit 14 performs a deformation detection process for detecting a deformation portion from each of the composite images IC _Pc1 , IC _Pc2 , . . . , IC _Pcn . Thereby, _the processing unit 14 generates deformation detection results RS _Pc1 , RS _Pc2 , .

The deformation detection result RS _Pc1 includes deformation data that is data of a deformation location detected from the composite image IC _Pc1 obtained using the parameter candidate Pc1. The deformation detection result RS _Pc2 includes deformation data that is data of a deformation location detected from the composite image IC _Pc2 obtained using the parameter candidate Pc2. Further, the deformation detection result RS _Pcn includes deformation data that is data of a deformation location detected from the composite image IC _Pcn obtained using the parameter candidate Pcn. In the _following _, when each of the deformation detection results RS _Pc1 , RS _Pc2 , .

Next, the processing unit 14 evaluates the deformation detection accuracy for the composite images IC _Pc1 , IC _Pc2 , ..., IC _Pcn based on the deformation detection results RS _Pc1 , RS _Pc2 , ..., RS _Pcn . Detection result evaluation processing for calculating values VR _Pc1 , VR _Pc2 , . . . , VR _Pcn is performed.

For example, the processing unit 14 calculates the evaluation value VR _Pc1 of the deformation detection accuracy for the composite image IC _Pc1 based on the deformation detection result RS _Pc1 , and calculates the _composite image IC An evaluation value VR _Pc2 of the deformation detection accuracy for _Pc2 is calculated. The processing unit 14 also calculates an evaluation value VR _Pcn of the deformation detection accuracy for the composite image IC _Pcn based on the deformation detection result RS _Pcn . . . _, VR _Pcn may be referred to as an evaluation value VR _Pc hereinafter when they are individually indicated without _being distinguished from each other. The evaluation value VR _Pc becomes a larger value as the deformation detection accuracy is higher.

Next, the processing unit 14 _, based on the evaluation values VR _Pc1 , VR _Pc2 , . decide. For example, the processing unit 14 determines the parameter candidate Pc corresponding to the largest evaluation value VR _Pc among the evaluation values VR _Pc1 , VR _Pc2 , _. process.

The processing unit 14 can also determine the parameter P for correcting the luminance based on the visibility of the composite image IC _Pc in addition to the detection accuracy of the deformed portion. The _{processing unit 14 calculates the visibility evaluation values VV Pc1} _, _VV _Pc2 _, _. . _. , VV _Pcn may be referred to as an evaluation value VV _Pc hereinafter when they are indicated without being distinguished from each other _.

The evaluation value VV _Pc is calculated based on the contrast and edge strength of the stitched image IC _Pc . For example, the processing unit 14 increases the evaluation value VV _Pc as the contrast increases, and increases the evaluation value VV _Pc as the edge strength increases.

The processing unit 14 determines, as the parameter P, the parameter candidate Pc corresponding to the largest result among the weighted addition results of the evaluation value VR _Pc and the evaluation value VV _Pc . For example, the processing unit 14 calculates a value obtained by weighting and adding the evaluation value VR _Pc1 and the evaluation value VV _Pc1 as the total evaluation value VT _Pc1 , and weights and adds the evaluation value VR _Pc2 and the evaluation value VV _Pc2 . The value is calculated as the comprehensive evaluation value VT _Pc2 .

In addition, the processing unit 14 calculates a total evaluation value VT _Pc by weighting and adding the evaluation value VR _Pcn and the evaluation value VV _Pcn . The processing unit 14 determines, as the parameter P, the parameter candidate Pc corresponding to the largest evaluation value among the comprehensive evaluation values VT _Pc1 , VT _Pc2 , . . . , VT _Pcn . . . _{, VT Pcn may be referred to as a total evaluation value VT Pc} _when _they are not individually distinguished _.

By determining the parameter P based on the comprehensive evaluation value VT _Pc , the processing unit 14 determines the parameter P for correcting the luminance based on the visibility of the stitched image IC _Pc in addition to the detection accuracy of the deformed portion. A parameter P can be determined. The processing unit 14 corrects the brightness of each of the captured images A, B, and C using the determined parameter P, and combines the captured images A, B, and C whose brightness has been corrected to generate a combined image _ICP . .

Then, the processing unit 14 detects a deformed portion from the composite image _ICP , and generates an inspection image in which information on the detected deformed portion is superimposed on the composite image _ICP . As a result, the processing unit 14 can improve the visibility of the composite image _ICP while suppressing deterioration in detection accuracy of the deformed portion from the composite image _ICP . The configuration of the processing unit 14 will be specifically described below.

As shown in FIG. 4, the processing unit 14 of the information processing apparatus 1 includes an input reception unit 30, a data acquisition unit 31, a luminance correction unit 32, an image stitching unit 33, a deformation detection unit 34, a parameter A determination unit 35 , a display processing unit 36 , an inspection image generation unit 37 , and a data output unit 38 are provided.

The input reception unit 30 receives an input operation to the input unit 11 by the user. For example, the input reception unit 30 receives a parameter determination request from the user via the input unit 11 . The parameter determination request includes, for example, information such as the object ID of the object whose parameters are to be determined.

Also, the input reception unit 30 receives an inspection image generation request from the user via the input unit 11 . The inspection image generation request includes, for example, information such as the object ID of the object whose parameters are to be determined. Also, the input reception unit 30 receives an output request from the user via the input unit 11 .

The data acquisition unit 31 acquires data from an external device via the communication unit 10 , stores the acquired data in the storage unit 13 , and causes the processing unit 14 to perform processing using the data stored in the storage unit 13 . The data stored in the storage unit 13 is acquired when it is performed.

For example, the data acquisition unit 31 acquires measurement data received by the communication unit 10 from the measuring device 70 via the network 3 from the communication unit 10, and stores captured image data of the acquired measurement data in the captured image data storage unit 20. be memorized.

Further, when the parameter determination request is received by the input receiving unit 30, the data acquisition unit 31 stores a plurality of captured image data and a plurality of parameter candidates Pc associated with the object ID included in the parameter determination request in the storage unit 13. Get from When the input reception unit 30 receives the parameter determination request, the operation mode of the processing unit 14 becomes the parameter determination mode.

Note that, when the parameter determination request is received by the input receiving unit 30, the data acquisition unit 31 stores a plurality of parameter candidates Pc associated with the object type of the object ID included in the parameter determination request in the storage unit 13. can also be obtained from

Further, when an inspection image generation request is received by the input reception unit 30, the data acquisition unit 31 acquires a plurality of captured image data and the parameter P associated with the object ID included in the inspection image generation request from the storage unit 13. get. When the input reception unit 30 receives the inspection image generation request, the operation mode of the processing unit 14 becomes the inspection image generation mode. The data acquisition unit 31 acquires the parameter P determined by the parameter determination unit 35 from the parameter determination unit 35 and stores it in the parameter storage unit 22 .

The brightness correction unit 32 has a parameter determination mode and an inspection image generation mode as operation modes. First, the operation of the brightness correction unit 32 when the operation mode is the parameter determination mode will be described.

When the operation mode is the parameter determination mode, the brightness correction unit 32 acquires the brightness of each of the plurality of captured images represented by the plurality of captured image data acquired by the data acquisition unit 31 . Correction is performed for each parameter candidate Pc using a plurality of parameter candidates Pc.

The luminance correction unit 32 corrects the luminance of each of the multiple captured images by gamma correction, for example. The brightness correction unit 32 performs gamma correction, for example, by calculating the following formula (1). In the following equation (1), "x" is the input, "y" is the output, and "γ" is the correction parameter.
f(x,y)=255×(x/255) ^1/γ (1)

In the case of gamma correction, parameter candidates Pc1, Pc2, . . . , Pcn are values to be substituted for parameter γ. The brightness correction unit 32 substitutes the parameter candidate Pc for “γ” in the above equation (1), and uses the brightness value of each pixel of the captured image as the input x, thereby obtaining the brightness of each of the plurality of captured images as a parameter candidate. It is corrected in units of Pc.

FIG. 9 is a diagram for explaining gamma correction performed by the luminance correction unit of the information processing apparatus according to the first embodiment. As shown in FIG. 9, when the parameter γ is less than 1, the larger the input x, the larger the output y increases. rate becomes smaller.

The luminance correction unit 32 can also correct the luminance of each of a plurality of captured images for each parameter candidate Pc by the local histogram equalization method. The local histogram equalization method is a method for improving contrast by dividing a captured image and correcting the luminance value distribution in each divided area.

In the case of the local histogram equalization method, the parameter candidates Pc1, Pc2, .

FIG. 10 is a diagram for explaining the local histogram equalization method executed by the luminance correction unit of the information processing apparatus according to the first embodiment. As shown in FIG. 10, in the local histogram equalization method, the contrast is improved by dividing the captured image and modifying the luminance value distribution in each divided area.

The luminance correction unit 32 can also correct the luminance of each of the plurality of captured images for each parameter candidate Pc using a high-pass filter. For example, the brightness correction unit 32 extracts the frequency components of the captured image by performing a discrete Fourier transform on the captured image, removes only the low-frequency components from the extracted frequency components, and produces a captured image from which only the low-frequency components are removed. By inverse Fourier transforming the frequency components of , a captured image with corrected brightness is generated. The brightness correction unit 32 can homogenize the image or extract edges by removing low-frequency components.

In the case of the high-pass filter, the parameter candidates Pc1, Pc2, . FIG. 11 is a diagram for explaining high-pass filtering performed by the luminance correction unit of the information processing apparatus according to the first embodiment;

In the example shown in FIG. 11, the parameter candidate Pc1 is a parameter specifying the removal frequency band fg1, and the parameter candidate Pc2 is a parameter specifying the removal frequency band fg2. Further, the parameter candidate Pc3 is a parameter specifying the removal frequency band fg3, and the parameter candidate Pc4 is a parameter specifying the removal frequency band fg4. The removal frequency band fg2 is higher than the removal frequency band fg1, the removal frequency band fg3 is higher than the removal frequency band fg2, and the removal frequency band fg4 is higher than the removal frequency band fg3.

Next, the operation of the luminance correction unit 32 when the operation mode is the inspection image generation mode will be described. When the operation mode is the inspection image generation mode, the brightness correction unit 32 acquires the brightness of each of the plurality of captured images represented by the plurality of captured image data acquired by the data acquisition unit 31. is corrected using the parameter P.

For example, the brightness correction unit 32 uses the parameter P to correct the brightness of each of the multiple captured images by gamma correction, local histogram equalization, or high-pass filtering.

The luminance correction method by the luminance correction unit 32 is not limited to gamma correction, local histogram equalization method, and high-pass filter, and various luminance correction methods can be applied. It may be a brightness correction method.

The luminance correction unit 32 can change the correction method according to the user's input operation to the input unit 11, or change the correction method according to the type of the target object. For example, the luminance correction unit 32 has correction information indicating a correction method corresponding to the type of object, and based on this correction information, a correction method for each object or a correction method corresponding to the type of object. can correct the luminance of each of a plurality of captured images. The type of object is, for example, the type of structure such as tunnel 4, guardrail 5, road, bridge, and the like. The luminance correction unit 32 can also change the correction method based on the time zone, season, weather, etc. when the captured image data was obtained.

Next, the image stitching unit 33 shown in FIG. 4 will be described. The image stitching unit 33 generates stitched images IC _Pc and _ICP , which are images obtained by stitching together a plurality of captured images whose luminance has been corrected by the luminance correcting unit 32 . The image stitching unit 33 has a parameter determination mode and an inspection image generation mode as operation modes. First, the operation of the image stitching unit 33 when the operation mode is the parameter determination mode will be described.

When the operation mode is the parameter determination mode, the image stitching unit 33 generates a stitched image IC Pc, which is an image obtained by stitching together a plurality of captured images whose brightness is corrected using the parameter candidates Pc in units of the parameter candidates _Pc . Generate.

For example, in the example shown in FIG. 8, the image stitching unit 33 stitches together a plurality of captured images A, B, and C whose brightness is corrected by the parameter candidates Pc1, Pc2, . . _. , IC _Pcn are generated _.

FIG. 12 is a diagram for explaining an example of captured image stitching processing by the image stitching unit of the information processing apparatus according to the first embodiment; In the example shown in FIG. 12, the image stitching unit 33 stitches together a plurality of captured images A, B, and C whose brightness is corrected by the parameter candidate Pc to generate a stitched image IC _Pc . Captured images A, B, and C shown in FIG. 12 are images of the inner wall surface 4 a of the tunnel 4 .

When the operation mode is the inspection image generation mode, the image stitching unit 33 generates a stitched image ICP, which is an image obtained by stitching together a plurality of captured images whose luminance has been corrected by the luminance correction unit 32 using the parameter _P. Generate.

Note that the image stitching unit 33 deletes the overlapping portion between the captured images A and B and the overlapping portion between the captured images B and C in the process of generating the stitched image IC _Pc or the stitched image _ICP .

Next, the deformation detector 34 shown in FIG. 4 will be described. The deformation detection unit 34 detects a deformed portion of the object using a learning model for detecting the deformed portion of the object from the composite images IC _Pc and _ICP . A learning model is a neural network such as a convolutional neural network or a recurrent neural network, and is generated by deep learning. The learning model of the deformation detection unit 34 is, for example, a learning model that performs semantic segmentation.

The deformation detection unit 34 has a parameter determination mode and an inspection image generation mode as operation modes. First, the operation of the deformation detector 34 when the operation mode is the parameter determination mode will be described.

When the operation mode is the parameter determination mode, the deformation detection unit 34 inputs the stitched image IC _Pc generated by the image stitching unit 33 to the learning model, and class label of each pixel output from the learning model. based on the score of

When the object is the tunnel 4, the class label is, for example, a class label indicating cracks, a class label indicating delamination, a class label indicating dirt, or a class label indicating water leakage. The class label score of a pixel is, for example, a value between 0 and 1. If there is a class label pixel with a score of 0.5 or more, the deformation detection unit 34 determines that the pixel is a deformation pixel. Yes, and the type of deformation is determined to be deformation corresponding to a class label with a score of 0.5 or higher.

For example, in the example shown in FIG. 8, the deformation detection unit 34 detects a deformation location from each of the composite images IC _Pc1 , _IC _Pc2 , . RS _Pc1 , RS _Pc2 , . . . , RS _Pcn are generated.

When the operation mode is the inspection image generation mode, the deformation detection unit 34 inputs the stitched image _ICP generated by the image stitching unit 33 to the learning model, and determines the class label of each pixel output from the learning model. Based on each score, the location of the deformation is detected.

Instead of a learning model that performs semantic segmentation, the learning model may be a learning model that is generated by labeling attributes of the entire composite images IC _Pc and IC _P. Also, the learning model may be a network model other than a neural network, or a computational model generated by machine learning other than deep learning, such as linear regression or logistic regression.

Next, the parameter determination unit 35 shown in FIG. 4 will be described. The parameter determination unit 35 determines the parameter P based on the detection accuracy of the deformation location detected by the deformation detection unit 34 whose operation mode is the parameter determination mode.

For example, in the example shown in FIG. 8, the parameter determining unit 35 calculates the deformation detection accuracy evaluation value VR _Pc for the composite image IC _Pc for each parameter candidate Pc based on the deformation detection result RS _Pc . Then, the parameter determining unit 35 determines the parameter P based on the evaluation value VR _Pc for each parameter candidate Pc. Thereby, the parameter determination unit 35 can determine the parameter P that can correct the luminance while suppressing deterioration in detection accuracy of the deformed portion of the object.

The parameter determination unit 35 can also determine the parameter P for correcting the brightness based on the visibility of the composite image IC _Pc in addition to the detection accuracy of the deformed portion. The parameter determining unit 35 calculates the visibility evaluation value VV _Pc of the composite image IC _Pc for each parameter candidate Pc.

Then, the parameter determination unit 35 calculates a total evaluation value VT _Pc for each parameter candidate Pc based on the evaluation value VR _Pc for the deformation detection accuracy and the evaluation value VV _Pc for visibility, and calculates the total evaluation value VT Pc for each parameter candidate Pc. A parameter P is determined based on the comprehensive evaluation value VT _Pc . Thereby, the parameter determination unit 35 can accurately determine the parameter P as a parameter capable of correcting the luminance while suppressing deterioration in detection accuracy of the deformed portion of the object.

The parameter determination unit 35 includes a true value information acquisition unit 40, an evaluation unit 41, and a determination unit 42, as shown in FIG. The true value information acquisition unit 40 acquires true value information including a true value indicating the presence or absence of deformation for each pixel of the composite image IC _Pc . When there are a plurality of types of deformation, the true value information includes a true value indicating the presence or absence of deformation for each type of deformation.

When the manual mode is set, the true value information acquisition unit 40 causes the display processing unit 36 to perform processing for displaying the composite image IC _Pc on the display unit 12, for example. The true value information acquiring unit 40 acquires the true value information accepted by the input accepting unit 30 via the data acquiring unit 31 when the true value information input by the user is accepted by the input accepting unit 30 .

In addition, when the automatic mode is set, the true value information acquisition unit 40 generates, in the image stitching unit 33, a stitched image obtained by stitching together a plurality of captured images for which luminance correction has not been performed by the luminance correction unit 32. Let Then, the true value information acquisition unit 40 causes the deformation detection unit 34 to detect a deformation location from a composite image obtained by combining a plurality of captured images that have not been subjected to luminance correction, and the deformation detection unit 34 detects the deformation location. The true value information is acquired from the information on the deformed location.

The evaluation unit 41 evaluates the composite image IC based on the information on the deformation location detected by the deformation detection unit 34 whose operation mode is the parameter determination mode and the true value information acquired by the true value information acquisition unit 40. An evaluation value VR _Pc of the detection accuracy of the deformed portion from _Pc is calculated.

The evaluation unit 41 calculates at least one of the recall rate, the precision rate, and the F value as the evaluation value VR _Pc , or weights and adds at least two of the recall rate, the precision rate, and the F value. This value can be calculated as the evaluation value VR _Pc .

Here, recall, precision, and F value will be explained. 13A and 13B are diagrams for explaining a method of calculating a recall rate, a precision rate, and an F value by an evaluation unit of the information processing apparatus according to the first embodiment; FIG.

In the example shown in FIG. 13, the number of pixels that are actually deformed out of the pixels detected as being deformed by the deformation detection unit 34 is TP (True Positive). The number of pixels that are actually background among pixels detected as being present is defined as TN (True Negative).

In the example shown in FIG. 13, the number of pixels that are actually deformed out of the pixels detected as background by the deformation detection unit 34 is FN (False Negative). FP (False Positive) is the number of pixels that are background among the pixels detected as having a shape. A background pixel is a pixel that indicates a non-deformation.

When there are multiple types of deformation, for example, the evaluation unit 41 treats deformation pixels that are not of the same type as background pixels, and calculates and totals TP, TN, FN, and FP for each deformation. , TP, TN, FN, and FP are calculated.

The evaluation unit 41 can calculate at least one of the recall rate, the precision rate, and the F value as the evaluation value VR _Pc by calculating the formula shown in FIG. The evaluation unit 41 can also calculate a value obtained by weighting and adding at least two of the recall rate, the precision rate, and the F value as the evaluation value VR _Pc .

In addition, the evaluation unit 41 calculates the difference between the score output from the learning model of the deformation detection unit 34 and the value corresponding to the actual pixel type for each pixel, and sums the calculated results as an evaluation value. It can also be VR _Pc . The value corresponding to the pixel type is "1" when the pixel type is deformation, and is "0" when the pixel type is background.

The evaluation unit 41 can also calculate the visibility evaluation value VV _Pc of the composite image IC _Pc based on the visibility of the composite image IC _Pc in addition to the detection accuracy of the deformed portion. The evaluation value VV _Pc is calculated based on the contrast and edge strength of the stitched image IC _Pc . For example, the evaluation unit 41 increases the evaluation value VV _Pc as the contrast increases, and increases the evaluation value VV _Pc as the edge strength increases.

In the example shown in _FIG _{. 8, the evaluation unit 41 calculates evaluation values VV Pc1} _, _VV _Pc2 , _. .

Then, the evaluation unit 41 calculates, for example, a value obtained by weighting and adding the evaluation value VR _Pc1 and the evaluation value VV _Pc1 as the total evaluation value VT _Pc1 , and weights the evaluation value VR _Pc2 and the evaluation value VV _Pc2 . The added value is calculated as the total evaluation value VT _Pc2 . The evaluation unit 41 also calculates a value obtained by weighting and adding the evaluation value VR _Pcn and the evaluation value VV _Pcn as the total evaluation value VT _Pcn .

The determination unit 42 shown in FIG. 4 determines the parameter P based on the multiple evaluation values VR _Pc or multiple evaluation values VV _Pc calculated by the evaluation unit 41 . Thereby, the determination unit 42 can determine the parameter P that can correct the brightness while suppressing deterioration in detection accuracy of the deformed portion of the object.

For example, the determination unit 42 determines, as the parameter P, the parameter candidate Pc corresponding to the largest evaluation value VR _Pc among the plurality of evaluation values VR _Pc calculated by the evaluation unit 41 . _For example _, _{it is assumed that the evaluation values VR Pc1} _, _VR _Pc2 _, . In this case, the determining unit 42 determines, as the parameter P, the parameter candidate Pc2 corresponding to the evaluation value VR _Pc2 .

Further, the determination unit 42 determines, as the parameter P, the parameter candidate Pc corresponding to the largest comprehensive evaluation value VT _Pc among the plurality of comprehensive evaluation values VT _Pc calculated by the evaluation unit 41 . _For _example _, _the evaluation unit 41 calculates the total evaluation values VT _Pc1 _, VT _Pc2 , . and In this case, the determining unit 42 determines, as the parameter P, the parameter candidate Pcn corresponding to the comprehensive evaluation value VT _Pcn . The parameter P determined by the determination unit 42 is stored in the parameter storage unit 22 by the data acquisition unit 31 as described above.

The display processing unit 36 causes the display unit 12 to display the information stored in the storage unit 13 or the information processed by the processing unit 14 on the display unit 12 based on the reception result by the input reception unit 30. or For example, the display processing unit 36 causes the display unit 12 to display the stitched image IC _Pc generated by the image stitching unit 33, or causes the display unit 12 to display the inspection image generated by the inspection image generation unit 37. do.

The inspection image generator 37 generates an inspection image by superimposing the stitched image _ICP generated by the image stitcher 33 and information on the deformed portion detected by the deformed detector 34 from the stitched image _ICP . do. The information on the deformed portion is, for example, a diagram showing the shape of the deformed portion, or a frame surrounding the deformed portion.

14 is a diagram of an example of an inspection image generated by the inspection image generation unit of the information processing apparatus according to the first embodiment; FIG. As shown in FIG. 14, in the inspection image generated by the inspection image generation unit 37, information highlighting the deformation detected by the deformation detection unit 34 is superimposed on the composite image _ICP . Although FIG. 14 shows the composite image _ICP in solid color, the composite image _ICP includes an image of the object.

The deformation is emphasized by setting the thickness or color of the line indicating the deformation to a thickness or color distinguishable from the composite image _ICP . The inspection image generator 37 can change the highlighting method according to the type of deformation and the type of object, for example. Note that the inspection image generation unit 37 can also output a composite image _ICP on which the information of the deformed portion is not superimposed as an inspection image.

When the input reception unit 30 receives an output request from the user, the data output unit 38 shown in FIG. is output to an external device via the communication unit 10 .

For example, the data output unit 38 outputs the parameter P determined by the parameter determination unit 35, the captured image data generated by the inspection image generation unit 37, and the like to an external device via the communication unit 10.

An external device to which the parameter P determined by the parameter determination unit 35 is transmitted is, for example, an information processing device that does not have the parameter determination unit 35, the parameter determination mode, and the parameter candidate storage unit 21. Such an external device acquires a plurality of imaged image data from the measurement device 70 of the measurement vehicle 2 or the like, corrects the brightness of each of the plurality of imaged images represented by the acquired plurality of imaged image data, and then corrects the brightness of each of the plurality of imaged images. A stitched image _ICP is generated by stitching together a plurality of captured images. Then, the external device detects a deformed portion from the composite image _ICP , and generates an inspection image in which information on the detected deformed portion is superimposed on the composite image _ICP .

Next, processing by the processing unit 14 of the information processing device 1 will be described using a flowchart. 15 is a flowchart illustrating an example of processing by a processing unit of the information processing apparatus according to the first embodiment; FIG.

As shown in FIG. 15, the processing unit 14 of the information processing device 1 determines whether measurement data output from the measurement device 70 of the measurement vehicle 2 or the like is acquired via the communication unit 10 (step S10). ). When the processing unit 14 determines that the measurement data has been acquired (step S10: Yes), the processing unit 14 stores captured image data included in the measurement data in the storage unit 13 (step S11).

When the processing of step S11 is completed, or when it is determined that the measurement data has not been acquired (step S10: No), the processing unit 14 determines whether or not the parameter determination timing has come (step S12). The processing unit 14 determines that the parameter determination timing has come, for example, when the user requests parameter determination or when the captured image data is stored in the storage unit 13 .

When the processing unit 14 determines that the parameter determination timing has come (step S12: Yes), it performs parameter determination processing (step S13). The process of step S13 is the process of steps S20 to S26 shown in FIG. 16, and will be described in detail later.

When the processing of step S13 is completed, or when it is determined that the parameter determination timing has not come (step S12: No), the processing unit 14 determines whether or not the inspection image generation timing has come (step S14). For example, when there is an inspection image generation request from the user, or after the parameter P is determined by the process of step S13, the processing unit 14 stores the captured image data of the object corresponding to the determined parameter P in the storage unit 13. , it is determined that the inspection image generation timing has come.

When the processing unit 14 determines that it is time to generate an inspection image (step S14: Yes), it executes inspection image generation processing (step S15). The process of step S15 is the process of steps S60 to S65 shown in FIG. 20, and will be described in detail later.

When the processing of step S15 is completed, or when it is determined that the inspection image generation timing has not come (step S14: No), the processing unit 14 determines whether or not there is an output request (step S16).

When the processing unit 14 determines that there is an output request (step S16: Yes), it performs output processing (step S17). In the output process of step S17, the processing unit 14 sends, for example, the parameter P determined by the parameter determination process of step S13 or the data of the inspection image generated by the inspection image generation process of step S15 to the external device via the communication unit 10. output.

When the processing of step S17 is finished, or when it is determined that there is no output request (step S16: No), the processing unit 14 determines whether or not it is time to end the operation (step S18). For example, when the processing unit 14 determines that the power supply (not shown) of the information processing device 1 is turned off or determines that an operation end operation has been performed on the input unit 11, the processing unit 14 determines that it is time to end the operation. do.

If the processing unit 14 determines that it is not the time to end the operation (step S18: No), the process proceeds to step S10. 15 ends.

16 is a flowchart illustrating an example of parameter determination processing by the processing unit of the information processing apparatus according to Embodiment 1. FIG. As shown in FIG. 16, the processing unit 14 acquires from the storage unit 13 a plurality of pieces of captured image data of an object to be subjected to parameter determination processing (step S20).

In addition, the processing unit 14 acquires from the storage unit 13 a plurality of parameter candidates Pc corresponding to the type of target object to be subjected to parameter determination processing or the target object (step S21). Then, the processing unit 14 uses each parameter candidate Pc to correct the brightness of the plurality of captured images represented by the plurality of captured image data (step S22).

Next, the processing unit 14 creates a composite image IC _Pc for each parameter candidate Pc by combining the plurality of captured images whose brightness has been corrected in step S22 for each parameter candidate Pc (step S23).

Next, the processing unit 14 detects a deformed portion for each parameter candidate Pc from the composite image IC _Pc (step S24). Then, the processing unit 14 executes a deformation detection accuracy evaluation process (step S25). The process of step S25 is the process of steps S30 to S32 shown in FIG. 17, and will be described in detail later.

Next, the processing unit 14 determines the parameter P used in the luminance correction process based on the evaluation value VR _Pc or the total evaluation value VT _Pc for each parameter candidate Pc calculated in the deformation detection accuracy evaluation process (step S26 ), the process shown in FIG. 16 is terminated.

FIG. 17 is a flowchart showing an example of deformation detection accuracy evaluation processing by the processing unit of the information processing apparatus according to the first embodiment. As shown in FIG. 17, the processing unit 14 determines whether or not the luminance correction process is set to the manual mode (step S30).

When the processing unit 14 determines that the luminance correction process is set to the manual mode (step S30: Yes), it executes the manual mode process (step S31). The process of step S31 is the process of steps S40 to S43 shown in FIG. 18, and will be described in detail later.

Also, when the processing unit 14 determines that the luminance correction process is set to the automatic mode instead of the manual mode (step S30: No), it executes the automatic mode process (step S32). The process of step S32 is the process of steps S50 to S56 shown in FIG. 19, and will be described in detail later.

The processing unit 14 ends the processing shown in FIG. 17 when the processing of step S31 is completed or when the processing of step S32 is completed.

18 is a flowchart showing an example of manual mode processing by the processing unit of the information processing apparatus according to Embodiment 1. FIG. As shown in FIG. 18, the processing unit 14 receives the true value of the deformed portion from the user (step S40).

Next, the processing unit 14 performs a process of comparing the detection result in step S24 and the true value received in step S40 for each pixel in parameter candidate Pc units, and calculates the detection accuracy evaluation value VR _Pc in parameter candidate Pc units. (step S41).

Next, the processing unit 14 calculates the visibility evaluation value VV _Pc of the composite image IC _Pc for each parameter candidate Pc (step S42). Then, the processing unit 14 calculates the total evaluation value VT _Pc for each parameter candidate Pc based on the detection accuracy evaluation value VR _Pc and the visibility evaluation value VV _Pc (step S43), and performs the process shown in FIG. exit.

19 is a flowchart showing an example of automatic mode processing by the processing unit of the information processing apparatus according to Embodiment 1. FIG. As shown in FIG. 19, the processing unit 14 acquires the same plurality of captured image data as the plurality of captured image data acquired in step S20 from the storage unit 13 (step S50).

Next, the processing unit 14 generates a composite image by combining a plurality of captured images represented by a plurality of captured image data acquired in step S50 without luminance correction (step S51). Then, the processing unit 14 detects a deformed portion from the composite image generated in step S51 (step S52).

Next, the processing unit 14 acquires true value information from the detection result of the deformed portion in step S52 (step S53). The processing unit 14 compares the detection result obtained in step S24 with the true value obtained in step S53 for each pixel, and calculates the evaluation value VR _Pc of the detection accuracy as the parameter candidate Pc. It is calculated in units (step S54).

Next, the processing unit 14 calculates the visibility evaluation value VV _Pc of the composite image IC _Pc for each parameter candidate Pc (step S55). Then, the processing unit 14 calculates the total evaluation value VT _Pc for each parameter candidate Pc based on the detection accuracy evaluation value VR _Pc and the visibility evaluation value VV _Pc (step S56), and performs the process shown in FIG. exit.

20 is a flowchart showing an example of inspection image generation processing by the processing unit of the information processing apparatus according to Embodiment 1. FIG. As shown in FIG. 20, the processing unit 14 acquires from the storage unit 13 a plurality of pieces of captured image data of an object to be inspected image generation processing (step S60).

Next, the processing unit 14 acquires from the storage unit 13 the type of object to be inspected image generation processing or the parameter P of the object (step S61). The processing unit 14 uses the parameter P acquired in step S61 to correct the brightness of the multiple captured images represented by the multiple captured image data acquired in step S60 (step S62).

Next, the processing unit 14 creates a composite image _ICP by combining a plurality of captured images whose brightness has been corrected (step S63). Then, the processing unit 14 detects a deformed portion from the composite image _ICP generated in step S63 (step S64).

Next, the processing unit 14 generates an inspection image by superimposing the information of the deformed portion detected in step S64 on the composite image _ICP generated in step S63 (step S65), and ends the processing shown in FIG. do.

FIG. 21 is a diagram illustrating an example of the hardware configuration of the information processing apparatus according to the first embodiment; As shown in FIG. 21, the information processing apparatus 1 includes a computer having a processor 101, a memory 102, a communication device 103, an input device 104, a display device 105, and a bus .

The processor 101, the memory 102, the communication device 103, the input device 104, and the display device 105 can transmit and receive information to and from each other via the bus 106, for example. Storage unit 13 is implemented by memory 102 . The communication unit 10 is implemented by the communication device 103 . The input unit 11 is implemented by the input device 104 . The display unit 12 is implemented by the display device 105 .

The processor 101 reads the information processing program from the recording medium set in the recording medium drive and installs the read information processing program in the memory 102 . The recording medium drive is, for example, a CD (Compact Disc)-ROM drive, a DVD (Digital Versatile Disc)-ROM drive, or a USB drive, and the recording medium is, for example, a CD-ROM, a DVD-ROM, or a non-volatile Such as semiconductor memory.

The processor 101 executes the functions of the processing unit 14 by reading and executing programs stored in the memory 102 . The processor 101 is an example of a processing circuit, for example, and includes one or more of a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a system LSI (Large Scale Integration).

The memory 102 includes one or more of RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), and EEPROM (registered trademark) (Electrically Erasable Programmable Read Only Memory). include. The information processing device 1 may include integrated circuits such as ASIC (Application Specific Integrated Circuit) and FPGA (Field Programmable Gate Array).

The information processing device 1 may be configured by a client device, may be configured by a server device, or may be configured by a client device and a server device. When the information processing apparatus 1 is composed of two or more devices, each of the two or more devices has the hardware configuration shown in FIG. 21, for example. Note that communication between two or more devices is performed via the communication device 103 . Further, the information processing device 1 may be composed of two or more server devices. For example, the information processing device 1 may be configured with a processing server and a data server.

In addition, the parameter determination unit 35 can also determine the parameter P individually for each of a plurality of captured images to be pasted together. In this case, the luminance correction unit 32 corrects the luminance of each of the plurality of captured images to be pasted together by combining different parameter candidates Pc.

As described above, the information processing apparatus 1 according to the first embodiment includes the data acquisition unit 31, the brightness correction unit 32, the image stitching unit 33, the deformation detection unit 34, and the parameter determination unit 35. . The data acquisition unit 31 acquires data of a plurality of images obtained by imaging an object with a plurality of

imaging devices

71a, 71b, and 71c. The luminance correction unit 32 corrects the luminance of each of the multiple images. The image stitching unit 33 generates a stitched image IC _Pc , which is an image obtained by stitching together a plurality of images whose luminance has been corrected by the luminance correcting unit 32 . The deformation detection unit 34 detects a deformed portion of the object using a learning model for detecting the deformed portion of the object from the composite image IC _Pc . The parameter determination unit 35 determines a parameter P used for luminance correction based on the accuracy of detection of the deformation location by the deformation detection unit 34 . As a result, the information processing apparatus 1 can appropriately determine the parameter P that can correct the luminance while suppressing deterioration in detection accuracy of the deformed portion of the object.

Further, the luminance correction unit 32 corrects the luminance of each of the plurality of images using each of the plurality of parameter candidates Pc. The image stitching unit 33 generates a stitched image IC _Pc , which is an image obtained by stitching together a plurality of images whose luminance has been corrected in units of parameter candidates Pc by the luminance correction unit 32 in units of parameter candidates Pc. The deformation detection unit 34 detects a deformation portion of the object in units of parameter candidates Pc from the composite image IC _Pc in units of parameter candidates Pc. The parameter determination unit 35 determines a parameter candidate Pc selected from a plurality of parameter candidates Pc as the parameter P based on the detection accuracy of the deformation location in units of the parameter candidates Pc by the deformation detection unit 34 . As a result, the information processing apparatus 1 can more appropriately determine the parameter P that can correct the brightness while suppressing deterioration in detection accuracy of the deformed portion of the object.

Also, the parameter determination unit 35 includes an evaluation unit 41 and a determination unit 42 . The evaluation unit 41 calculates an evaluation value VR _Pc of the detection accuracy of the deformation location for each parameter candidate Pc by the deformation detection unit 34 . The determination unit 42 determines a parameter candidate Pc to be used as the parameter P from among the plurality of parameter candidates Pc based on the evaluation value VR _Pc calculated by the evaluation unit 41 . As a result, the information processing apparatus 1 can more appropriately determine the parameter P that can correct the brightness while suppressing deterioration in detection accuracy of the deformed portion of the object.

In addition, the evaluation unit 41 further calculates the visibility evaluation value VV _Pc of the composite image ICP for each parameter candidate _Pc , and performs a comprehensive evaluation based on the detection accuracy evaluation value VR _Pc and the visibility evaluation value _VVPc . A value VT _Pc is calculated for each parameter candidate Pc. The determination unit 42 determines a parameter candidate Pc to be the parameter P from among the plurality of parameter candidates Pc based on the comprehensive evaluation value VT _Pc evaluated by the evaluation unit 41 . As a result, the information processing apparatus 1 can more appropriately determine the parameter P that can correct the brightness while suppressing deterioration in detection accuracy of the deformed portion of the object.

Also, the brightness correction unit 32 corrects the brightness of each of a plurality of images obtained by imaging the object with the plurality of

imaging devices

71 a , 71 b , 71 c using the parameter P determined by the parameter determination unit 35 . The image stitching unit 33 generates a stitched image _ICP , which is an image obtained by stitching together a plurality of images whose luminance is corrected by the parameter P by the luminance correcting unit 32 . The deformation detection unit 34 detects a deformation portion of the object from the composite image _ICP whose luminance is corrected by the parameter P. FIG. As a result, the information processing apparatus 1 can correct the luminance while suppressing deterioration in detection accuracy of the deformed portion of the object.

The configuration shown in the above embodiment is an example, and can be combined with another known technique, and part of the configuration can be omitted or changed without departing from the scope of the invention. It is possible.

1 information processing device, 2 measurement vehicle, 3 network, 4 tunnel, 4a inner wall surface, 5 guardrail, 10, 74 communication unit, 11 input unit, 12 display unit, 13 storage unit, 14, 73 processing unit, 20 captured image data Storage unit 21 Parameter candidate storage unit 22 Parameter storage unit 30 Input reception unit 31 Data acquisition unit 32 Brightness correction unit 33 Image stitching unit 34 Deformation detection unit 35 Parameter determination unit 36 Display processing unit , 37 inspection image generation unit, 38 data output unit, 40 true value information acquisition unit, 41 evaluation unit, 42 determination unit, 60 vehicle body, 70 measurement device, 71, 71a, 71b, 71c imaging device, 72 position/attitude speed detection Unit, 100 measurement processing system, 101 processor, 102 memory, 103 communication device, 104 input device, 105 display device, 106 bus.

Claims

a data acquisition unit that acquires data of a plurality of images obtained by imaging an object with a plurality of imaging devices;
a luminance correction unit that corrects the luminance of each of the plurality of images;
an image stitching unit that generates a stitched image that is an image obtained by stitching together the plurality of images whose luminance has been corrected by the luminance correction unit;
a deformation detection unit that detects a deformed portion of the object using a learning model that detects the deformed portion of the object from the combined image;
an information processing apparatus comprising: a parameter determination unit that determines a parameter used for correcting the luminance based on the accuracy of detection of the deformation location by the deformation detection unit.
The luminance correction unit
correcting the brightness of each of the plurality of images using each of the plurality of parameter candidates;
The image stitching unit includes:
generating a stitched image obtained by stitching together the plurality of images whose luminance has been corrected in units of the candidate parameters by the luminance correction unit in units of the candidate parameters;
The deformation detection unit is
detecting a deformed portion of the object in units of the parameter candidates from the stitched image in units of the parameter candidates;
The parameter determination unit
2. The parameter candidate according to claim 1, wherein a parameter candidate selected from among the plurality of parameter candidates is determined as the parameter based on detection accuracy of the deformed portion in the parameter candidate unit by the deformation detection unit. information processing equipment.
The parameter determination unit
an evaluation unit that calculates an evaluation value of detection accuracy of the deformation location for each of the parameter candidates by the deformation detection unit;
3. The information according to claim 2, further comprising a determination unit that determines a parameter candidate to be used as the parameter from among the plurality of parameter candidates based on the evaluation value calculated by the evaluation unit. processing equipment.
The evaluation unit
further calculating a visibility evaluation value of the stitched image for each of the parameter candidates, calculating a comprehensive evaluation value based on the detection accuracy evaluation value and the visibility evaluation value for each of the parameter candidates;
The decision unit
4. The information processing apparatus according to claim 3, wherein a parameter candidate to be said parameter is determined from among said plurality of parameter candidates based on said comprehensive evaluation value evaluated by said evaluation unit.
The luminance correction unit
correcting the brightness of each of a plurality of images obtained by imaging an object with the plurality of imaging devices using the parameter determined by the parameter determining unit;
The image stitching unit includes:
generating a stitched image that is an image obtained by stitching together the plurality of images, the luminance of which has been corrected by the luminance correction unit using the parameter;
The deformation detection unit is
5. The information processing apparatus according to any one of claims 1 to 4, wherein a deformed portion of the object is detected from the composite image in which the luminance is corrected by the parameter.
A computer-executed information processing method comprising:
a data acquisition step of acquiring data of a plurality of images obtained by imaging an object with a plurality of imaging devices;
a brightness correction step of correcting the brightness of each of the plurality of images;
an image stitching step of generating a stitched image that is an image obtained by stitching together the plurality of images whose luminance has been corrected by the luminance correction step;
a deformation detection step of detecting a deformed portion of the object using a learning model for detecting a deformed portion of the object from the combined image;
and a parameter determination step of determining a parameter used for correcting the brightness based on the detection accuracy of the deformed portion obtained by the deformation detection step.
a data acquisition step of acquiring data of a plurality of images obtained by imaging an object with a plurality of imaging devices;
a brightness correction step of correcting the brightness of each of the plurality of images;
an image stitching step of generating a stitched image that is an image obtained by stitching together the plurality of images whose luminance has been corrected by the luminance correction step;
a deformation detection step of detecting a deformed portion of the object using a learning model for detecting a deformed portion of the object from the combined image;
and a parameter determination step of determining a parameter used for correcting the brightness based on the detection accuracy of the deformed portion obtained by the deformation detection step.