WO2021176621A1 - 画像処理装置、画像処理方法、および画像処理プログラム - Google Patents

画像処理装置、画像処理方法、および画像処理プログラム Download PDF

Info

Publication number
WO2021176621A1
WO2021176621A1 PCT/JP2020/009273 JP2020009273W WO2021176621A1 WO 2021176621 A1 WO2021176621 A1 WO 2021176621A1 JP 2020009273 W JP2020009273 W JP 2020009273W WO 2021176621 A1 WO2021176621 A1 WO 2021176621A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
detection
detection target
unit
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/009273
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
高之 天見
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Original Assignee
Mitsubishi Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp filed Critical Mitsubishi Electric Corp
Priority to PCT/JP2020/009273 priority Critical patent/WO2021176621A1/ja
Priority to JP2020544680A priority patent/JP6793896B1/ja
Publication of WO2021176621A1 publication Critical patent/WO2021176621A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination

Definitions

  • the present disclosure relates to an image processing device, an image processing method, and an image processing program that masks an area of an image of a structure that does not detect deformation of a detection target.
  • a technique for detecting a deformation occurring in a structure from an image of a structure has been known. May be detected.
  • the image of the tunnel includes areas other than cracks, such as lighting, cables, free lime marks, dirt, or formwork marks. , These areas may be mistakenly detected as cracks.
  • Patent Document 1 proposes a technique for generating an image in which a crack non-detection region, which is a region in which cracks are not detected, is masked in an image of a structure.
  • the technique described in Patent Document 1 uses features such as an illumination area, a cable area, a free lime area, a dirt area, or a formwork mark area, and a set threshold value or a numerical range to image a tunnel.
  • the crack non-detection region is determined from the inside.
  • the technique described in Patent Document 1 masks the determined crack non-detection region in the tunnel image, and detects the crack from the masked tunnel image.
  • the present disclosure has been made in view of the above, and is an image processing capable of masking a non-detection region which is a region in which a deformation of a detection target is not detected without adjusting a threshold value or a numerical range.
  • the purpose is to obtain the device.
  • the image processing apparatus of the present disclosure includes an acquisition unit, a non-detection target detection unit, and a masking unit.
  • the acquisition unit acquires information on the image of the structure.
  • the non-detection target detection unit has a structure acquired by the acquisition unit using one or more detection models that detect a region of the non-detection target that is generated by machine learning and is a target different from the deformation of the detection target from the image.
  • the non-detection target area is detected from the image of the object.
  • the masking unit masks the non-detection target area detected by the non-detection target detection unit in the image of the structure.
  • the non-detection region which is the region where the deformation of the detection target is not detected, without adjusting the threshold value or the numerical range.
  • FIG. 1 is a diagram showing an example of an image processing method executed by the image processing apparatus according to the first embodiment.
  • the image processing device 1 shown in FIG. 1 detects a specific type of deformation occurring in the structure from the image of the structure as a detection target.
  • the image processing device 1 detects a non-detection region, which is a region in the image of the structure that does not detect deformation of the detection target, using the detection model generated by machine learning, and masks the detected non-detection region. do.
  • the image processing device 1 detects a specific type of deformation as a detection target from the image of the structure in which the non-detection region is masked.
  • the image of the structure processed by the image processing device 1 is, for example, a developed image of the inner wall surface of a tunnel, an image of the surface of a bridge, an image of the surface of a road, or an image of the surface of a building.
  • the structure whose image is processed by the image processing device 1 may be a structure other than a tunnel, a bridge, a road, and a building. In the following, the image of the structure processed by the image processing device 1 will be described as an image of the inner wall surface of the tunnel.
  • the specific type of deformation detected as a detection target by the image processing device 1 is a crack, but in addition to the crack, leaked water or the like may be included.
  • the masked non-detection area is, for example, an illumination area, a bolt area, a cable area, a free lime area, a dirty area, a formwork mark area, or a repair mark area.
  • each of lighting, bolts, cables, free lime, dirt, formwork marks, and repair marks may be described as non-detection targets.
  • the image processing method executed by the image processing device 1 will be described with reference to FIG.
  • information on an image of a structure is input to the image processing device 1.
  • the image of such a structure is a developed image of the inner wall surface of the tunnel when it is assumed that the surface of the inner wall surface of the tunnel is developed in a two-dimensional plane.
  • the developed image of the inner wall surface of the tunnel is, for example, an image obtained by pasting together images taken multiple times by each of the plurality of cameras while a measuring vehicle equipped with a plurality of cameras is traveling in the tunnel. Is.
  • the developed image of the tunnel shown in FIG. 1 includes images of repair marks, lighting, and cracks.
  • the developed image of the tunnel is a monochrome image represented by brightness, but may be a color image.
  • the image processing device 1 performs detection processing on the image of the structure.
  • a detection process is a process for detecting a non-detection target area to be masked, and is a process executed by using a detection model generated by machine learning.
  • the detection model is a calculation model generated by machine learning and detecting a non-detection target area from an image, and when an image of a structure is input, information on the non-detection target area is output.
  • the output of the detection model includes information indicating that there is no non-detection target region.
  • Such a detection model is generated by, for example, deep learning.
  • the image processing device 1 detects the non-detection target area by using the detection model generated for each type of the non-detection target. For example, the image processing device 1 detects an illumination area using a detection model for detecting illumination, detects a bolt area using a detection model for detecting bolts, and detects a cable. Detect the area of the cable using a detection model. Further, the image processing apparatus 1 detects a free lime region using a detection model for detecting free lime, detects a stain region using a detection model for detecting stains, and creates formwork marks. The area of the formwork mark is detected using a detection model for detection.
  • the image processing device 1 detects the area of the repair mark by using the detection model for detecting the repair mark.
  • the image processing device 1 may be configured to detect each non-detection target using a detection model common to all non-detection targets, and may use a detection model common to two or more non-detection targets. It may be configured to detect the detection target.
  • the image processing device 1 detects the non-detection target area by the detection process, and then masks the non-detection target area in the image of the structure as the non-detection area.
  • Masking by the image processing device 1 is performed, for example, by replacing a non-detection target area in the image of the structure with a preset image.
  • the preset image is, for example, a monochromatic image such as a black image or a white image.
  • the preset image may be an image in which the deformation of the detection target is not detected, and is not limited to a monochromatic image.
  • the masking by the image processing device 1 can also be performed by associating the information indicating the non-detection target region in the image of the structure with the information of the image of the structure.
  • the image processing device 1 performs deformation detection processing for detecting a specific type of deformation from an image of a masked structure.
  • the image processing device 1 associates the information indicating the non-detection target area with the information of the image of the structure, the image processing device 1 refers to the area other than the non-detection target area based on the information indicating the non-detection target area. Performs processing to detect a specific type of deformation.
  • the image processing device 1 detects the non-detection target region in the image of the structure by using the detection model generated by machine learning. Therefore, the image processing device 1 can mask the non-detection target area without adjusting the threshold value or the numerical range, and can accurately detect a specific type of deformation.
  • the image processing apparatus 1 will be described in more detail.
  • FIG. 2 is a diagram showing an example of the configuration of the image processing apparatus according to the first embodiment.
  • the image processing device 1 according to the first embodiment includes a communication unit 10, an input unit 11, a storage unit 12, and a processing unit 13.
  • the communication unit 10 is connected to a network such as a LAN (Local Area Network) or a WAN (Wide Area Network), and transmits / receives information to and from an external device via the network.
  • the input unit 11 is, for example, a mouse, a keyboard, or a touch panel, and is operated by the user of the image processing device 1.
  • the storage unit 12 includes an image information storage area 20, a non-detection target information storage area 21, a deformed information storage area 22, and a learning information storage area 23.
  • the image information storage area 20 stores information on the structure image, which is an image of the structure acquired by the processing unit 13 from the external device via the communication unit 10.
  • FIG. 3 is a diagram showing an example of a structure image according to the first embodiment.
  • the structure image 2 also includes formwork marks.
  • repair marks 9 1, 9 2, 9 3, 9 4, 9 if each of 5 are shown without individually distinguished, may be referred to as repair marks 9.
  • the non-detection target information storage area 21 stores the non-detection target information which is the information of the non-detection target detected from the structure image 2 by the processing unit 13.
  • deformation information which is information of a specific type of deformation detected from the structure image 2 by the processing unit 13 is stored.
  • the learning information storage area 23 stores learning information for generating a detection model for detecting a non-detection target area in the structure image 2.
  • the learning information includes a plurality of images of the non-detection target and a plurality of images other than the non-detection target for each non-detection target.
  • the learning information includes a plurality of images of the illumination 4 and a plurality of images other than the illumination 4 as information for generating a detection model for detecting the illumination 4.
  • the learning information includes a plurality of images of the repair mark 9 and a plurality of images other than the repair mark 9 as information for generating a detection model for detecting the repair mark 9.
  • the image of the non-detection target is, for example, an image of an area surrounded by a circumscribed rectangle which is a rectangle circumscribing the non-detection target while surrounding the non-detection target.
  • the image to be undetected does not have to be an image of an area surrounded by an extrinsic rectangle, and may be, for example, an image of an area surrounded by a frame other than a rectangle.
  • the learning information of each non-detection target includes information on a plurality of images of the non-detection target for each of the color of the inner wall surface of the tunnel, the type of tunnel construction method, and the presence or absence of a panel provided on the inner wall of the tunnel. Information on a plurality of images other than the non-detection target is included.
  • the processing unit 13 includes an acquisition unit 30, a learning unit 31, an information receiving unit 32, a non-detection target detection unit 33, a masking unit 34, and a deformation detection unit 35.
  • the acquisition unit 30 acquires the information transmitted from the external device (not shown) and received by the communication unit 10 from the communication unit 10, and stores the acquired information in the storage unit 12.
  • the acquisition unit 30 acquires the information of the structure image 2 transmitted from the external device (not shown) and received by the communication unit 10 from the communication unit 10, and the acquired information of the structure image 2 is stored in the image information storage area 20. To memorize. Further, the acquisition unit 30 acquires the learning information transmitted from the external device (not shown) and received by the communication unit 10 from the communication unit 10, and stores the acquired learning information in the learning information storage area 23.
  • the learning unit 31 reads learning information from the learning information storage area 23 of the storage unit 12, and uses a detection model that detects each non-detection target based on the read learning information, such as the surface color of the structure and the structure. It is generated for each of the method of construction and the presence or absence of a panel in the structure. Hereinafter, it is assumed that the structure is a tunnel.
  • the non-detection target is lighting 4, bolt 5, cable 6, free lime 7, dirt 8, formwork mark, or repair mark 9, but other targets may be non-detection targets, and some of them may be non-detection targets. It may be a non-detection target.
  • the tunnel will be repaired with wire mesh or filler. When the tunnel is repaired with wire mesh, the repair mark 9 is a wire mesh, and when the tunnel is repaired with a filler, the repair mark 9 is a filler.
  • the learning unit 31 uses each of the detection models of lighting 4, bolt 5, cable 6, free lime 7, dirt 8, formwork mark, or repair mark 9 as the color of the inner wall surface of the tunnel, the type of tunnel construction method, and the tunnel construction method. Generated for each presence or absence of panels provided on the inner wall of the tunnel.
  • each of the detection model of the illumination 4, the detection model of the bolt 5, the detection model of the cable 6, the detection model of the free lime 7, the detection model of the dirt 8, the detection model of the mold mark, and the detection model of the repair mark 9. Is generated for each of the color of the inner wall of the tunnel, the type of construction method of the tunnel, and the presence or absence of panels provided on the inner wall of the tunnel.
  • the color of the inner wall surface of the tunnel is an example of the surface color of the structure
  • the construction method of the tunnel is an example of the construction method of the structure
  • the panel provided on the inner wall of the tunnel is an example of the panel in the structure.
  • the detection model for detecting each non-detection target is generated for any one or two of the information on the surface color of the structure, the information on the construction method of the structure, and the information on the presence or absence of the panel in the structure. May be good. Further, the detection model for detecting each non-detection target may be one without being divided according to the information on the surface color of the structure, the information on the construction method of the structure, and the information on the presence or absence of the panel in the structure.
  • the learning unit 31 can also generate a detection model common to all non-detection targets.
  • the type of color of the inner wall surface of the tunnel is, for example, light gray, gray, or dark gray.
  • the type of color of the inner wall surface of the tunnel may be two or four or more. Since the detection model is generated for each color of the inner wall surface of the tunnel, the non-detection target detection unit 33 can accurately detect the non-detection target.
  • the type of construction method for the structure is, for example, a shield method, a TBM (Tunnel Boring Machine) method, or a mountain method. Deformation that occurs in the structure may differ depending on the type of construction method. Since the learning unit 31 generates a detection model for each construction method of the structure, the non-detection target detection unit 33 can accurately detect the non-detection target.
  • the types of construction methods for structures may include construction methods other than these construction methods.
  • the panel in the structure is, for example, an interior panel provided on the inner wall surface of the tunnel. Further, on the surface of the interior panel, for example, a reflective member such as a reflector or a reflective tape that reflects light is provided. Since the learning unit 31 generates a detection model when there is a panel provided on the inner wall of the tunnel and a detection model when there is no panel provided on the inner wall of the tunnel, the non-detection target detection unit 33 accurately determines the non-detection target. It can be detected well.
  • the above-mentioned repair mark 9 detection model includes, for example, a wire mesh detection model and a filler detection model.
  • the wire mesh detection model is a detection model that detects the area repaired by the wire mesh.
  • the filler detection model is a detection model that detects the area repaired by the filler.
  • the detection model generated by the learning unit 31 is a calculation model that detects a non-detection target region that is generated by machine learning and is a target different from the detection target crack 3 from the image, and is generated by, for example, deep learning.
  • the detection model is a neural network such as a convolutional neural network, but may be a calculation model generated by a learning algorithm such as linear regression or logistic regression, or may be a calculation model such as a support vector machine. good.
  • the detection model of the non-detection target generated by the learning unit 31 outputs the information of the detected non-detection target area, and in addition to the information of the detected non-detection target area, the detected non-detection target area. It may be configured to output a label.
  • the label of the non-detection target is information indicating the type of the non-detection target. For example, in the case of the detection model of the illumination 4, the information indicating that the non-detection target is the illumination 4.
  • the information reception unit 32 receives structure information, which is information about the structure. For example, the information receiving unit 32 receives the structure information by the operation of the input unit 11 by the user. Further, the information receiving unit 32 receives the structure information transmitted from an external device (not shown) and received by the communication unit 10.
  • the structure information includes, for example, at least one of information on the surface color of the structure, information on the construction method of the structure, and information on the presence or absence of a panel in the structure.
  • the structure information may include information on the repair method of the structure. Types of repair methods include repair with wire mesh and repair with filler.
  • the non-detection target detection unit 33 uses a plurality of detection models corresponding to the structure information received by the information reception unit 32 among the plurality of detection models generated by machine learning and detecting the non-detection target region from the image. Then, the non-detection target area is detected from the structure image 2.
  • the non-detection target detection unit 33 detects the non-detection target region from the structure image 2 by using a plurality of detection models associated with the combination of light gray, the shield method, and the presence of the panel.
  • the non-detection target detection unit 33 uses a detection model corresponding to the specified tunnel repair method included in the structure information to form a structure. The area of the repair mark 9 in the object image 2 is detected.
  • the non-detection target detection unit 33 detects the area of the repair mark 9, which is the area repaired by the wire mesh, by using the wire mesh detection model. .. Further, when the repair method included in the structure information is repair by the filler, the non-detection target detection unit 33 uses the filler detection model to use the filler detection model to repair the region of the repair mark 9 which is the region repaired by the filler. Is detected.
  • the non-detection target detection unit 33 is a detection model capable of detecting both the region repaired by the wire mesh and the region repaired by the filler when the structure information does not include information on the tunnel repair method. Can be used to detect the area of the repair mark 9. Such a detection model is also generated by the learning unit 31 for each of the types of tunnel construction methods and the presence or absence of panels provided on the inner wall of the tunnel.
  • the plurality of detection models corresponding to the structure information are the detection model of lighting 4, the detection model of bolt 5, the detection model of cable 6, the detection model of free lime 7, the detection model of dirt 8, the detection model of formwork marks, and the like. And includes a detection model for repair marks 9.
  • the non-detection target detection unit 33 uses the corresponding detection model among the plurality of detection models to detect each region of the illumination 4, the bolt 5, the cable 6, the free lime 7, the dirt 8, the formwork mark 9, and the repair mark 9. To detect.
  • the non-detection target detection unit 33 outputs the non-detection target information, which is the detected non-detection target information, to the masking unit 34, and further stores the non-detection target information in the non-detection target information storage area 21.
  • the non-detection target information includes information on the detected non-detection target ID (Identifier), information indicating the type of the detected non-detection target, and coordinates of the structure image 2 in which the detected non-detection target region exists. Including information.
  • the ID of the non-detection target is an ID assigned by the deformation detection unit 35, and is a different ID for each non-detection target.
  • the coordinates are, for example, the coordinates of an orthogonal coordinate system having one of the four corners of the structure image 2 as the origin.
  • the non-detection target area is, for example, a rectangular area
  • the coordinate information in which the non-detection target area exists is, for example, information on the coordinates of two diagonal coordinates of the four corners of the rectangle, or a rectangle. It is the information of the coordinates of the four corners of.
  • the non-detection target detection unit 33 detects the illumination 4, the non-detection target ID, the information indicating that the non-detection target is the illumination 4, and the region of the illumination 4 in the structure image 2 exist.
  • the non-detection target information including the information of the coordinates to be used is output to the masking unit 34 and stored in the non-detection target information storage area 21.
  • the non-detection target detection unit 33 detects the repair mark 9, the ID of the non-detection target, the information indicating that the non-detection target is the repair mark 9, and the repair mark 9 in the structure image 2
  • the non-detection target information including the information of the coordinates where the area exists is output to the masking unit 34 and stored in the non-detection target information storage area 21.
  • FIG. 4 is a diagram showing an example of each non-detection target region detected by the image processing apparatus according to the first embodiment.
  • the non-detection target detection unit 33 includes a region 41 of the illumination 4, a region 42 of the bolt 5, a region 43 of the cable 6, a region 44 of the free lime 7, and a region 45 of the dirt 8.
  • the region 46 of the repair mark 9 is detected.
  • Each of the areas 41, 42, 43, 44, 45, and 46 detected by the non-detection target detection unit 33 is a rectangular area.
  • the non-detection target detection unit 33 detects, for example, a region surrounded by a rectangle circumscribing the non-detection target while surrounding the non-detection target as a non-detection target region. In the example shown in FIG. 4, the area of the formwork mark is not shown, but the non-detection target detection unit 33 also detects the area of the formwork mark.
  • the non-detection target detection unit 33 acquires the color information of the inner wall surface of the tunnel from the structure information, but detects the color of the inner wall surface of the tunnel from the structure image 2 and detects the tunnel. It is also possible to detect a non-detection target by using a plurality of detection models corresponding to the color of the inner wall surface of the tunnel.
  • the structure information includes information indicating the type of tunnel construction method, but the structure information may be information indicating the shape of the tunnel.
  • the non-detection target detection unit 33 determines the type of tunnel construction method from the information indicating the shape of the tunnel, and detects the non-detection target using a plurality of detection models corresponding to the determined type of tunnel construction method. do.
  • the masking unit 34 reads the structure image information which is the information of the structure image 2 from the image information storage area 20 of the storage unit 12, and the read structure image information and the non-detection target detected by the non-detection target detection unit 33.
  • the non-detection target area of the structure image 2 is masked based on the information of.
  • Masking by the masking unit 34 is performed, for example, by replacing the non-detection target area in the structure image 2 with a preset image.
  • the preset image is, for example, a monochromatic image such as a black image or a white image.
  • the preset image may be an image in which the deformation of the detection target is not detected, and is not limited to a monochromatic image.
  • masking by the masking unit 34 can also be performed by associating the non-detection target information, which is information indicating the non-detection target region of the structure image 2, with the information of the structure image 2.
  • the masking unit 34 performs masking by associating the ID of the structure image 2 with the ID of each non-detection target.
  • the deformation detection unit 35 detects the crack 3 as a specific type of deformation from the masked structure image 2.
  • the deformation detection unit 35 detects the coordinates, length, and width of the detected crack 3.
  • the deformation detection unit 35 stores the deformation information including the coordinate, length, and width information of the detected crack 3 and the ID information of the detected crack 3 in the deformation information storage area 22.
  • the ID of the crack 3 is an ID assigned by the deformation detection unit 35, and is a different ID for each of the cracks 3.
  • the deformation detected by the deformation detection unit 35 may include leaked water or the like in addition to cracks.
  • the detection model is generated by the image processing device 1, but the detection model used by the non-detection target detection unit 33 of the image processing device 1 is performed by a learning device different from the image processing device 1. May be good.
  • the acquisition unit 30 of the image processing device 1 can acquire a plurality of detection models generated by a learning device different from the image processing device 1 from the learning device via the communication unit 10.
  • the acquisition unit 30 outputs a plurality of acquired detection models to the non-detection target detection unit 33.
  • the non-detection target detection unit 33 detects the non-detection target region from the structure image 2 based on the plurality of detection models output from the acquisition unit 30.
  • FIG. 5 is a flowchart showing an example of processing by the processing unit of the image processing apparatus according to the first embodiment.
  • the processing unit 13 of the image processing device 1 repeatedly executes the processing shown in FIG.
  • the processing unit 13 performs a learning process (step S1).
  • the learning process of step S1 is the process of steps S10 to S15 shown in FIG. 6, which will be described in detail later.
  • the processing unit 13 performs a crack detection process (step S2).
  • the crack detection process in step S2 is the process of steps S20 to S29 shown in FIG. 7, which will be described in detail later.
  • FIG. 6 is a flowchart showing an example of learning processing by the processing unit of the image processing apparatus according to the first embodiment.
  • the acquisition unit 30 of the processing unit 13 determines whether or not the learning information has been acquired from an external device (not shown) (step S10).
  • the acquisition unit 30 determines that the learning information has been acquired (step S10: Yes)
  • the acquisition unit 30 stores the acquired learning information in the storage unit 12 (step S11).
  • the learning unit 31 of the processing unit 13 determines whether or not the learning timing has been reached when the processing of step S11 is completed or when the acquisition unit 30 determines that the learning information has not been acquired (step S10: No). Determine (step S12). In the process of step S12, the learning unit 31 of the processing unit 13 reaches the learning timing when, for example, the learning information is stored in the storage unit 12, or when there is a learning request due to an operation on the input unit 11.
  • step S12 determines that the learning timing has come (step S12: Yes)
  • the learning unit 31 acquires learning information from the storage unit 12 (step S13).
  • the learning unit 31 generates a detection model for each non-detection target based on the learning information acquired in step S13 (step S14).
  • the learning unit 31 outputs the generated detection model information for each non-detection target to the non-detection target detection unit 33 (step S15).
  • step S15 the learning unit 31 generates a plurality of detection models for each of, for example, the color of the inner wall surface of the tunnel, the type of the tunnel construction method, and the presence or absence of the panel provided on the inner wall of the tunnel.
  • the processing unit 13 ends the processing shown in FIG. 6 when the processing in step S15 is completed or when it is determined that the learning timing has not been reached (step S12: No).
  • FIG. 7 is a flowchart showing an example of crack detection processing by the processing unit of the image processing apparatus according to the first embodiment.
  • the acquisition unit 30 of the processing unit 13 determines whether or not the information of the structure image 2 has been acquired from an external device (not shown) (step S20).
  • the acquisition unit 30 determines that the information of the structure image 2 has been acquired (step S20: Yes)
  • the acquisition unit 30 stores the acquired information of the structure image 2 in the storage unit 12 (step S21).
  • step S21 Whether or not the information receiving unit 32 of the processing unit 13 has received the structure information when the processing of step S21 is completed or when it is determined that the information of the structure image 2 has not been acquired (step S20: No). Is determined (step S22). In the process of step S22, when the operation to the input unit 11 by the user is the input operation of the structure information, the information receiving unit 32 determines that the structure information has been received.
  • step S22 When the non-detection target detection unit 33 of the processing unit 13 determines that the structure information has been received by the information reception unit 32 (step S22: Yes), a plurality of non-detection target detection units 33 according to the structure information received by the information reception unit 32. A detection model is selected (step S23). The non-detection target detection unit 33 detects a non-detection target region in the structure image 2 by using a plurality of selected detection models (step S24).
  • the non-detection target detection unit 33 of the processing unit 13 stores the detected information of each non-detection target in the storage unit 12 (step S25).
  • the masking unit 34 of the processing unit 13 masks the non-detection target area of the structure image 2 based on the detected information of each non-detection target (step S26).
  • the deformation detection unit 35 of the processing unit 13 detects the region of the crack 3 from the masked structure image 2 (step S27).
  • the deformation detection unit 35 detects the coordinates, length, and width of the detected crack 3 (step S28), and stores the detected information of the crack 3 in the storage unit 12 (step S29).
  • step S28 the process of step S29 is completed, or when the information receiving unit 32 determines that the structure information is not received (step S22: No), the processing unit 13 ends the process shown in FIG. 7.
  • FIG. 8 is a diagram showing an example of the hardware configuration of the image processing device according to the first embodiment.
  • the image processing device 1 includes a computer including a processor 101, a memory 102, an input device 103, and a communication device 104.
  • the processor 101, the memory 102, the input device 103, and the communication device 104 can send and receive information to and from each other by, for example, the bus 105.
  • the communication unit 10 is realized by the communication device 104.
  • the input unit 11 is realized by the input device 103.
  • the storage unit 12 is realized by the memory 102.
  • the processor 101 reads and executes the acquisition unit 30, the learning unit 31, the information reception unit 32, the non-detection target detection unit 33, the masking unit 34, the deformation detection unit 35, and the like. Perform the function of.
  • the processor 101 is, for example, an example of a processing circuit, and includes one or more of a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a system LSI (Large Scale Integration).
  • the memory 102 is one or more of RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), and EEPROM (registered trademark) (Electrically Erasable Programmable Read Only Memory). include.
  • the memory 102 also includes a recording medium on which a computer-readable program is recorded. Such recording media include one or more of non-volatile or volatile semiconductor memories, magnetic disks, flexible memories, optical disks, compact disks, and DVDs (Digital Versatile Discs).
  • the image processing device 1 may include integrated circuits such as an ASIC (Application Specific Integrated Circuit) and an FPGA (Field Programmable Gate Array).
  • the image processing device 1 includes an acquisition unit 30, a non-detection target detection unit 33, and a masking unit 34.
  • the acquisition unit 30 acquires the information of the structure image 2.
  • the non-detection target detection unit 33 is acquired by the acquisition unit 30 using one or more detection models that detect a region of the non-detection target that is generated by machine learning and is a target different from the deformation of the detection target from the image.
  • a non-detection target area is detected from the structure image 2.
  • the masking unit 34 masks the non-detection target area detected by the non-detection target detection unit 33 in the structure image 2.
  • the image processing apparatus 1 can mask the non-detection region without adjusting the threshold value or the numerical range.
  • the image processing device 1 includes an information receiving unit 32 that receives structure information, which is information about the structure.
  • the non-detection target detection unit 33 uses one or more detection models corresponding to the structure information received by the information reception unit 32 among the plurality of detection models generated by machine learning and detecting the non-detection target region from the image. It is used to detect a non-detection target region from the structure image 2. As a result, the image processing device 1 can narrow down a more appropriate detection model based on the structure information, and can accurately detect the non-detection target region.
  • the structure information includes at least one of information on the surface color of the structure, information on the construction method of the structure, and information on the presence or absence of a panel in the structure.
  • the image processing apparatus 1 can narrow down an appropriate detection model with at least one of the information on the surface color of the structure, the information on the construction method of the structure, and the information on the presence or absence of the panel, and the non-detection target. The area can be detected accurately.
  • the structure information includes information on how to repair the structure.
  • the non-detection target detection unit 33 uses one or more detection models that detect a region repaired by the structure repair method specified by the structure information among a plurality of detection models as a non-detection target, and uses a non-detection target. Detect the area of. As a result, the image processing device 1 can use an appropriate detection model according to the repair method, and can accurately detect the region of the repair mark 9.
  • the image processing device 1 includes a learning unit 31 that generates one or more detection models by machine learning based on learning information. As a result, the image processing device 1 can generate a detection model.
  • Embodiment 2 The image processing apparatus according to the second embodiment outputs information on the drawing of the structure including information indicating the position of the detected non-detection target area, and when the position of the non-detection target area is changed, the image processing apparatus outputs the information of the drawing of the structure. It differs from the image processing apparatus 1 according to the first embodiment in that the detection model is relearned.
  • components having the same functions as those of the first embodiment are designated by the same reference numerals and description thereof will be omitted, and the differences from the image processing device 1 of the first embodiment will be mainly described.
  • FIG. 9 is a diagram showing an example of the configuration of the image processing apparatus according to the second embodiment.
  • the image processing device 1A according to the second embodiment includes a display unit 14, and also includes a storage unit 12A and a processing unit 13A in place of the storage unit 12 and the processing unit 13. It is different from the processing device 1.
  • the display unit 14 is, for example, an LCD (Liquid Crystal Display), an OLED (Organic Electro-Luminescence Display), or an IELD (Inorganic Electro-Luminescence Display).
  • LCD Liquid Crystal Display
  • OLED Organic Electro-Luminescence Display
  • IELD Inorganic Electro-Luminescence Display
  • the storage unit 12A is different from the storage unit 12 in that it further has a drawing information storage area 25.
  • Information on the drawing of the structure is stored in the drawing information storage area 25.
  • the information of the drawing of the structure is, for example, the information of the development drawing of the tunnel.
  • the processing unit 13A is different from the processing unit 13 in that it includes a learning unit 31A instead of the learning unit 31, and further includes a drawing information generation unit 36, a drawing information output unit 37, and a drawing information changing unit 38.
  • the drawing information generation unit 36 generates the structure drawing information which is the information of the structure drawing when the user of the image processing device 1A requests the drawing display by the input operation to the input unit 11.
  • the structure drawing information is information on the drawing of the structure including information indicating the position of each non-detection target area detected by the non-detection target detection unit 33 and information indicating the type of the non-detection target.
  • the information indicating the type of the non-detection target includes, for example, information indicating the name of the lighting 4, information indicating the name of the bolt 5, information indicating the name of the cable 6, information indicating the name of the free lime 7, and the name of the dirt 8.
  • the drawing information generation unit 36 acquires information indicating the position and type of each non-detection target area detected by the non-detection target detection unit 33 from the non-detection target information storage area 21, and provides information on the drawing of the structure. Is acquired from the drawing information storage area 25.
  • the drawing information generation unit 36 generates structure drawing information by superimposing information indicating each of the positions and types of each non-detection target area on the drawing of the structure, for example.
  • the drawing information output unit 37 outputs the structure drawing information generated by the drawing information generation unit 36 to the display unit 14, and causes the display unit 14 to display the structure drawing.
  • FIG. 10 is a diagram showing an example of a structure drawing displayed on a display unit of the image processing apparatus according to the second embodiment.
  • the structure drawing 50 shown in FIG. 10 shows an image of a rectangular frame showing an area of each non-detection target detected by the non-detection target detection unit 33, and a name of each non-detection target associated with each frame. Contains information.
  • the structure drawing 50 shown in FIG. 10 includes an image and a name of a rectangular frame showing the positions of the lighting 4, the bolt 5, the cable 6, the free lime 7, the dirt 8, and the repair mark 9.
  • the structure drawing information generated by the drawing information generation unit 36 may be image information obtained by superimposing the structure image 2 and the image shown in FIG. 10 on the drawing of the structure.
  • the user of the image processing device 1A can change the position of the non-detection target area included in the structure drawing 50 displayed on the display unit 14 by the input operation to the input unit 11.
  • the drawing information changing unit 38 changes the position of the non-detection target area detected by the non-detection target detection unit 33 when there is a request to change the position of the non-detection target area included in the structure drawing 50. do.
  • the drawing information changing unit 38 receives a request to change the position of the non-detection target area included in the structure drawing 50. Judge that there was.
  • the position of the non-detection target area is changed by changing the non-detection target information stored in the non-detection target information storage area 21.
  • the drawing information changing unit 38 indicates the position of the cable 6 among the non-detection target information stored in the non-detection target information storage area 21. Change the non-detection target information including.
  • the drawing information changing unit 38 extracts the image of the position of the changed non-detection target area from the structure image 2. Then, the extracted image is added to the learning information corresponding to the non-detection target changed as the correct image. For example, when the position of the cable 6 is changed, the drawing information changing unit 38 extracts an image of the changed position of the cable 6 from the structure image 2, and detects the cable 6 by using the extracted image of the cable 6 as the correct image. Add to the training information to generate the model.
  • the learning unit 31A can relearn the detection model using the learning information updated by the drawing information changing unit 38. As a result, the image processing device 1A can improve the accuracy of the detection model.
  • the drawing information generation unit 36 uses, for example, information indicating the position of the deformed region detected by the deformation detection unit 35 and information indicating the position of each non-detection target region as information on the drawing of the structure. By superimposing, the structure drawing information can also be generated. In this case, the drawing information generation unit 36 acquires the deformation information from the deformation information storage area 22. Then, the drawing information changing unit 38 detects cracks detected by the deformation detecting unit 35 when, for example, there is a request to change the coordinates, length, and width of the crack to be detected included in the structure drawing 50. Change the coordinates, length, and width of.
  • FIG. 11 is a flowchart showing an example of processing by the processing unit of the image processing apparatus according to the second embodiment.
  • the processing unit 13A of the image processing device 1A repeatedly executes the processing shown in FIG. Since the processes of steps S30 and S31 shown in FIG. 11 are the same as the processes of steps S1 and S2 shown in FIG. 5, the description thereof will be omitted.
  • step S32 the processing unit 13A of the image processing device 1A performs a drawing correction process (step S32).
  • step S32 is the process of steps S40 to S44 shown in FIG. 12, which will be described in detail later.
  • the processing unit 13A performs a re-learning process (step S33).
  • step S33 is the process of steps S50 to S52 shown in FIG. 13, which will be described in detail later.
  • the processing unit 13A ends the process shown in FIG.
  • FIG. 12 is a flowchart showing an example of drawing correction processing by the processing unit of the image processing apparatus according to the second embodiment.
  • the processing unit 13A of the image processing device 1A determines whether or not there is a drawing display request from the user of the image processing device 1A (step S40).
  • the processing unit 13A determines that there is a drawing display request (step S40: Yes)
  • the processing unit 13A generates the structure drawing information, and causes the display unit 14 to display the structure drawing 50 based on the generated structure drawing information. (Step S41).
  • step S41 determines whether or not there is a drawing correction request from the user of the image processing device 1A (step S40: No).
  • step S42 determines whether or not there is a drawing correction request from the user of the image processing device 1A (step S40: No).
  • step S42 determines that there is a drawing correction request (step S42: Yes)
  • step S43 the processing unit 13A updates the non-detection target information based on the drawing correction request
  • step S44 updates the learning information based on the updated non-detection target information.
  • step S44 When the processing in step S44 is completed or when it is determined that there is no drawing correction request (step S42: No), the processing unit 13A ends the processing shown in FIG.
  • FIG. 13 is a flowchart showing an example of the re-learning process by the processing unit of the image processing device according to the second embodiment.
  • the processing unit 13A of the image processing device 1A determines whether or not the re-learning timing has come (step S50).
  • the processing unit 13A determines that the re-learning timing has come when the learning information is updated in the process of step S44.
  • the processing unit 13A may determine that the re-learning timing has come when it is determined that the re-learning request has been made by the user's operation on the input unit 11 instead of the timing for updating the learning information. can.
  • step S50 determines that the re-learning timing has come (step S50: Yes)
  • the processing unit 13A acquires the updated learning information from the storage unit 12A (step S51), and does not detect it based on the acquired learning information.
  • a target detection model is generated (step S52).
  • step S52 When the process of step S52 is completed or when it is determined that the re-learning timing has not been reached (step S50: No), the processing unit 13A ends the process shown in FIG.
  • the hardware configuration example of the image processing device 1A according to the second embodiment is the same as the hardware configuration of the image processing device 1 shown in FIG.
  • the processor 101 reads and executes the acquisition unit 30, the learning unit 31A, the information reception unit 32, the non-detection target detection unit 33, the masking unit 34, the deformation detection unit 35, and the drawing.
  • the functions of the information generation unit 36, the drawing information output unit 37, and the drawing information change unit 38 can be executed.
  • the image processing device 1A includes a drawing information generation unit 36, a drawing information output unit 37, and a drawing information changing unit 38.
  • the drawing information generation unit 36 generates the structure drawing information which is the information of the structure drawing 50 including the information indicating the position of the non-detection target area detected by the non-detection target detection unit 33.
  • the drawing information output unit 37 outputs the structure drawing information generated by the drawing information generation unit 36.
  • the drawing information changing unit 38 changes the position of the non-detection target area detected by the non-detection target detection unit 33 when there is a request to change the position of the non-detection target area included in the structure drawing information. do.
  • the learning unit 31A relearns one or more detection models based on the information of the image of the non-detection target area changed by the drawing information changing unit 38 in the structure image 2. As a result, the image processing device 1A can improve the accuracy of the detection model.
  • the configuration shown in the above embodiments is an example, and can be combined with another known technique, can be combined with each other, and does not deviate from the gist. It is also possible to omit or change a part of the configuration.
  • 1,1A image processing apparatus 2 building image, 3,3 1, 3 2, 3 3, 3 4, 3 5 cracks, 4,4 1, 4 2, 4 3, 4 4, 4 5, 4 6, 4 7 , 4 8 Lighting, 5, 5 1 , 5 2 Volts, 6 Cables, 7 Free lime, 8 Dirt, 9, 9 1 , 9 2 , 9 3 , 9 4 , 9 5 Repair marks, 10 Communication parts, 11 Input unit, 12, 12A storage unit, 13, 13A processing unit, 14 display unit, 20 image information storage area, 21 non-detection target information storage area, 22 deformation information storage area, 23 learning information storage area, 25 drawing information Storage area, 30 Acquisition unit, 31, 31A Learning unit, 32 Information reception unit, 33 Non-detection target detection unit, 34 Masking unit, 35 Deformity detection unit, 36 Drawing information generation unit, 37 Drawing information output unit, 38 Drawing information Changes, 41, 42, 43, 44, 45, 46 areas, 50 structure drawings.

Landscapes

  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Image Analysis (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
PCT/JP2020/009273 2020-03-04 2020-03-04 画像処理装置、画像処理方法、および画像処理プログラム Ceased WO2021176621A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/JP2020/009273 WO2021176621A1 (ja) 2020-03-04 2020-03-04 画像処理装置、画像処理方法、および画像処理プログラム
JP2020544680A JP6793896B1 (ja) 2020-03-04 2020-03-04 画像処理装置、画像処理方法、および画像処理プログラム

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/009273 WO2021176621A1 (ja) 2020-03-04 2020-03-04 画像処理装置、画像処理方法、および画像処理プログラム

Publications (1)

Publication Number Publication Date
WO2021176621A1 true WO2021176621A1 (ja) 2021-09-10

Family

ID=73544674

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/009273 Ceased WO2021176621A1 (ja) 2020-03-04 2020-03-04 画像処理装置、画像処理方法、および画像処理プログラム

Country Status (2)

Country Link
JP (1) JP6793896B1 (https=)
WO (1) WO2021176621A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7344620B1 (ja) * 2022-07-22 2023-09-14 株式会社 Sai 建屋内構造物認識システム及び建屋内構造物認識方法

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2023129993A (ja) * 2022-03-07 2023-09-20 オムロン株式会社 外観検査システム、外観検査方法、及びプログラム
KR102911215B1 (ko) * 2022-03-29 2026-01-13 민팃(주) 전자 기기의 외관 상태 평가 및 가치 평가 방법과 전자 기기 가치 평가 장치

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016121953A (ja) * 2014-12-25 2016-07-07 西日本高速道路エンジニアリング四国株式会社 トンネル覆工面画像のひび割れ領域抽出のための画像処理方法
WO2019053842A1 (ja) * 2017-09-14 2019-03-21 三菱電機株式会社 変状検出装置
WO2019163329A1 (ja) * 2018-02-21 2019-08-29 富士フイルム株式会社 画像処理装置及び画像処理方法

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109374631B (zh) * 2018-09-30 2021-06-29 中国铁道科学研究院集团有限公司铁道建筑研究所 一种隧道状态评测方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2016121953A (ja) * 2014-12-25 2016-07-07 西日本高速道路エンジニアリング四国株式会社 トンネル覆工面画像のひび割れ領域抽出のための画像処理方法
WO2019053842A1 (ja) * 2017-09-14 2019-03-21 三菱電機株式会社 変状検出装置
WO2019163329A1 (ja) * 2018-02-21 2019-08-29 富士フイルム株式会社 画像処理装置及び画像処理方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7344620B1 (ja) * 2022-07-22 2023-09-14 株式会社 Sai 建屋内構造物認識システム及び建屋内構造物認識方法
WO2024018642A1 (ja) * 2022-07-22 2024-01-25 株式会社 Sai 建屋内構造物認識システム及び建屋内構造物認識方法

Also Published As

Publication number Publication date
JP6793896B1 (ja) 2020-12-02
JPWO2021176621A1 (https=) 2021-09-10

Similar Documents

Publication Publication Date Title
JP6793896B1 (ja) 画像処理装置、画像処理方法、および画像処理プログラム
US11408740B2 (en) Map data update apparatus, map data update method, and computer readable medium
JP7118277B2 (ja) 検査装置、検査方法及びプログラム
KR100280132B1 (ko) 참조 화상 작성 방법 및 패턴 검사 장치
TWI757585B (zh) 檢查裝置、檢查方法及檢查程式
US6883160B2 (en) Pattern inspection apparatus
JPH0835936A (ja) プリント回路基板の自動光学検査装置における歪み補正方法及び装置
JP6749528B1 (ja) 変状管理装置、変状管理方法、および変状管理プログラム
US20230296531A1 (en) Information processing apparatus, information processing method, and information processing program
JP7273748B2 (ja) 検査装置、検査方法、及びプログラム
CN110614631B (zh) 确定目标点的方法、装置、电子设备及计算机可读介质
US7275006B2 (en) Workpiece inspection apparatus assisting device, workpiece inspection method and computer-readable recording media storing program therefor
JP2009294027A (ja) パターン検査装置及び方法
KR20180061511A (ko) 표시 결함 보상 방법과 이를 이용한 액정 표시 장치
JP2009092551A (ja) 障害物計測方法、障害物計測装置及び障害物計測システム
JP7194410B2 (ja) 診断方法、プログラム、および診断装置
JP4233556B2 (ja) 画像補正装置、パターン検査装置、画像補正方法、及び、パターン検査方法
US9410897B2 (en) Film edge detecting method and film edge detecting device
CN117314824B (zh) 掩模板颗粒的检测方法、检测装置、电子设备及存储介质
JP5566257B2 (ja) データ生成方法および画像検査方法
JP5245509B2 (ja) パターンデータの処理方法、及び電子デバイスの製造方法
JP4817861B2 (ja) パターン寸法測定方法及び寸法測定装置
CN113345017A (zh) 一种使用标志辅助视觉slam的方法
JP3122178B2 (ja) パターン検査装置
KR20210095068A (ko) 얼라인먼트 마크 검출 장치 및 얼라인먼트 마크 검출 방법

Legal Events

Date Code Title Description
ENP Entry into the national phase

Ref document number: 2020544680

Country of ref document: JP

Kind code of ref document: A

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20922948

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20922948

Country of ref document: EP

Kind code of ref document: A1