KR102441272B1 - Method for measuring crack based on artificial intelligence and apparatus thereof - Google Patents

Abstract

The present invention relates to a method for measuring cracks. Various embodiments of the present invention relate to the method and apparatus for analyzing cracks based on artificial intelligence. The present invention includes a memory and a processor connected to the memory, The processor obtains information related to a first image of a specific wall surface among a plurality of wall surfaces included in a structure.

Description

Method and apparatus for measuring cracks based on artificial intelligence

The present invention relates to a method for measuring cracks.

Various embodiments of the present invention are directed to a method and apparatus for analyzing cracks based on artificial intelligence.

In general, as a damage investigation and inspection method performed on a concrete structure, a method of visually inspecting by an inspector using a survey tool directly on the site is generally used. Since this is highly dependent on the subjectivity of the inspector, the objectivity and reliability of the records are lowered. Therefore, a new method that can automatically detect cracks and objectively analyze crack characteristics is required.

Korean Patent Registration No. 10-2016531

Various embodiments may provide methods and apparatus for analyzing cracks based on artificial intelligence.

Technical problems to be achieved in various embodiments are not limited to those mentioned above, and other technical problems not mentioned are considered by those of ordinary skill in the art from various embodiments to be described below. can be

According to various embodiments, a method performed by an electronic device may be provided.

According to various embodiments, the method includes: obtaining information related to a first image of a specific wall among a plurality of walls included in a structure; and analysis information including one or more analysis results for one or more cracks included in the specific wall surface based on output information obtained in response to input of the input information including the first image into a preset crack analysis model. acheive; may include doing

According to various embodiments, as the input information is input, a plurality of sub-images having a preset size based on the first image may be acquired by the preset crack analysis model.

According to various embodiments, the information related to the first image may include location information on the location of the specific wall surface.

According to various embodiments, the analysis information may include location coordinates corresponding to the specific wall identified based on the location information in one or more of the second images related to the structure and the one or more analysis results mapped may contain information.

According to various embodiments, the one or more analysis results may include information on a length of each of the one or more cracks and information on a thickness of each of the one or more cracks.

According to various embodiments, the length of each of the one or more cracks may be determined by the preset crack analysis model: a crack image included in each of the plurality of sub-images and a specific wall image are distinguished; the crack image is associated with at least some of the one or more cracks; the specific wall image is associated with at least a portion of the specific wall surface; obtaining a plurality of crack pixels for at least some of the one or more cracks from the crack image; a plurality of sub-crack pixels are obtained for obtaining a length of at least a portion of the one or more cracks among the plurality of crack pixels; a first starting pixel corresponding to a first starting point of the one or more cracks and a first ending pixel corresponding to a first ending point of the one or more cracks are identified from among the plurality of sub-crack pixels; and calculating a length of at least a portion of the one or more cracks based on the sequential tracking of the plurality of sub-crack pixels from the first start pixel to the first end pixel; can be obtained based on

According to various embodiments, the thickness of each of the one or more cracks may be obtained by the preset crack analysis model: a maximum value among a plurality of candidate thicknesses corresponding to each of the plurality of sub-crack pixels.

According to various embodiments, the plurality of candidate thicknesses are determined according to the preset crack analysis model: a specific sub-crack pixel (Pn) among the plurality of sub-crack pixels as an endpoint and the specific sub-crack pixel (Pn) A first vector having the immediately preceding first sub-crack pixel (Pn-1) as a starting point and a second sub-cracking pixel (Pn+) immediately after the specific sub-cracking pixel (Pn) with the specific sub-cracking pixel (Pn) as a starting point 1) a normal vector at the specific sub-crack pixel Pn, which is the difference between the second vectors with , is obtained; Based on the tracking of at least some of the plurality of sub-crack pixels along the direction of the normal vector, a second start pixel corresponding to a second start point of the one or more cracks and a first end point of the one or more cracks a corresponding second end pixel is identified; and calculating a candidate thickness corresponding to the specific sub-cracked pixel (Pn) based on the interval between the second start pixel and the second end pixel. , and n may be a natural number.

According to various embodiments, the preset crack analysis model may be preset based on application of machine learning to an artificial intelligence (AI) engine for obtaining the preset crack analysis model.

According to various embodiments, the machine learning comprises: - (a) learning the AI engine based on training data for the preset crack analysis model; - (b) obtaining feedback information on processing data output in response to input of test data for verification of the learned AI engine into the AI engine; - (c) updating the AI engine based on the feedback information; and - (d) (a) to (c) are repeated, but each time (a) to (c) is repeated, the count value having an initial value of 0 is incremented by 1, and the count value is a preset count threshold exit based on matching a value; It can be performed based on

According to various embodiments, the training data and the test data may be obtained based on training data for the preset crack analysis model.

According to various embodiments, the training data includes: a plurality of wall front images, a plurality of wall non-frontal images, a plurality of wall crack images for a wall crack, and an image for candidates that are likely to be confused with the wall crack may include

According to various embodiments, the candidates likely to be confused with the wall crack may include: an engraved decoration on the wall and a joint on the wall.

According to various embodiments, the second image may include a plurality of structure images for a plurality of floors included in the structure.

According to various embodiments, the location information may include information on a layer including the specific wall surface among the plurality of layers.

According to various embodiments, a structure image of a layer including the specific wall surface among the plurality of structural images may be identified based on information on the layer including the specific wall surface.

According to various embodiments, a structure image of a layer including the specific wall surface, position coordinates corresponding to the specific wall surface, and the one or more analysis results may be mapped.

According to various embodiments, the location coordinates may be two-dimensional coordinates based on a preset reference coordinate system for the location of the specific wall surface on the structural image of the layer including the specific wall surface.

According to various embodiments, based on an angle between a first normal perpendicular to the surface of the first image and a second normal perpendicular to the specific wall surface obtained from the first image is less than or equal to a preset threshold: Sub-images of , may be a segmentation of the first image into the preset size.

According to various embodiments, based on the angle between the first normal and the second normal exceeding the preset threshold: according to the preset crack analysis model, the angle between the first normal and the second normal is The first image may be corrected to be less than or equal to the preset threshold, and the plurality of sub-images may be obtained by dividing the first image corrected by the preset crack analysis model into the preset size.

According to various embodiments, an electronic device may be provided.

According to various embodiments, the electronic device may include: a memory; and one or more processors connected to the memory.

According to various embodiments, the one or more processors may: acquire information related to a first image of a specific wall among a plurality of walls included in the structure; and analysis information including one or more analysis results for one or more cracks included in the specific wall surface based on output information obtained in response to input of the input information including the first image into a preset crack analysis model. acheive; can be set to

According to various embodiments, as the input information is input, a plurality of sub-images having a preset size based on the first image may be acquired by the preset crack analysis model.

According to various embodiments, the information related to the first image may include location information on the location of the specific wall surface.

According to various embodiments, the analysis information may include location coordinates corresponding to the specific wall identified based on the location information in one or more of the second images related to the structure and the one or more analysis results mapped may contain information.

According to various embodiments, the one or more analysis results may include information on a length of each of the one or more cracks and information on a thickness of each of the one or more cracks.

According to various embodiments, the length of each of the one or more cracks may be determined by the preset crack analysis model: a crack image included in each of the plurality of sub-images and a specific wall image are distinguished; the crack image is associated with at least some of the one or more cracks; the specific wall image is associated with at least a portion of the specific wall surface; obtaining a plurality of crack pixels for at least some of the one or more cracks from the crack image; a plurality of sub-crack pixels are obtained for obtaining a length of at least a portion of the one or more cracks among the plurality of crack pixels; a first starting pixel corresponding to a first starting point of the one or more cracks and a first ending pixel corresponding to a first ending point of the one or more cracks are identified from among the plurality of sub-crack pixels; and calculating a length of at least a portion of the one or more cracks based on the sequential tracking of the plurality of sub-crack pixels from the first start pixel to the first end pixel; can be obtained based on

According to various embodiments, the thickness of each of the one or more cracks may be obtained by the preset crack analysis model: a maximum value among a plurality of candidate thicknesses corresponding to each of the plurality of sub-crack pixels.

According to various embodiments, the plurality of candidate thicknesses are determined according to the preset crack analysis model: a specific sub-crack pixel (Pn) among the plurality of sub-crack pixels as an endpoint and the specific sub-crack pixel (Pn) A first vector having the immediately preceding first sub-crack pixel (Pn-1) as a starting point and a second sub-cracking pixel (Pn+) immediately after the specific sub-cracking pixel (Pn) with the specific sub-cracking pixel (Pn) as a starting point 1) a normal vector at the specific sub-crack pixel Pn, which is the difference between the second vectors with , is obtained; Based on the tracking of at least some of the plurality of sub-crack pixels along the direction of the normal vector, a second start pixel corresponding to a second start point of the one or more cracks and a first end point of the one or more cracks a corresponding second end pixel is identified; and calculating a candidate thickness corresponding to the specific sub-cracked pixel (Pn) based on the interval between the second start pixel and the second end pixel. , and n may be a natural number.

According to various embodiments, the preset crack analysis model may be preset based on application of machine learning to an artificial intelligence (AI) engine for obtaining the preset crack analysis model.

According to various embodiments, the machine learning comprises: - (a) learning the AI engine based on training data for the preset crack analysis model; - (b) obtaining feedback information on processing data output in response to input of test data for verification of the learned AI engine into the AI engine; - (c) updating the AI engine based on the feedback information; and - (d) (a) to (c) are repeated, but each time (a) to (c) is repeated, the count value having an initial value of 0 is incremented by 1, and the count value is a preset count threshold exit based on matching a value; It can be performed based on

According to various embodiments, the training data and the test data may be obtained based on training data for the preset crack analysis model.

According to various embodiments, the training data includes: a plurality of wall front images, a plurality of wall non-frontal images, a plurality of wall crack images for a wall crack, and an image for candidates that are likely to be confused with the wall crack may include

According to various embodiments, the candidates likely to be confused with the wall crack may include: an engraved decoration on the wall and a joint on the wall.

According to various embodiments, the second image may include a plurality of structure images for a plurality of floors included in the structure.

According to various embodiments, the location information may include information on a layer including the specific wall surface among the plurality of layers.

According to various embodiments, a structure image of a layer including the specific wall surface among the plurality of structural images may be identified based on information on the layer including the specific wall surface.

According to various embodiments, a structure image of a layer including the specific wall surface, position coordinates corresponding to the specific wall surface, and the one or more analysis results may be mapped.

According to various embodiments, the location coordinates may be two-dimensional coordinates based on a preset reference coordinate system for the location of the specific wall surface on the structural image of the layer including the specific wall surface.

According to various embodiments, based on an angle between a first normal perpendicular to the surface of the first image and a second normal perpendicular to the specific wall surface obtained from the first image is less than or equal to a preset threshold: Sub-images of , may be a segmentation of the first image into the preset size.

According to various embodiments, based on the angle between the first normal and the second normal exceeding the preset threshold: according to the preset crack analysis model, the angle between the first normal and the second normal is The first image may be corrected to be less than or equal to the preset threshold, and the plurality of sub-images may be obtained by dividing the first image corrected by the preset crack analysis model into the preset size.

According to various embodiments, a non-transitory processor-readable medium storing one or more instructions to cause one or more processors to perform an operation will be provided. can

According to various embodiments, the operation may include: acquiring information related to a first image of a specific wall among a plurality of walls included in the structure; and analysis information including one or more analysis results for one or more cracks included in the specific wall surface based on output information obtained in response to input of the input information including the first image into a preset crack analysis model. acheive; may include doing

According to various embodiments, as the input information is input, a plurality of sub-images having a preset size based on the first image may be acquired by the preset crack analysis model.

According to various embodiments, the information related to the first image may include location information on the location of the specific wall surface.

According to various embodiments, the analysis information may include location coordinates corresponding to the specific wall identified based on the location information in one or more of the second images related to the structure and the one or more analysis results mapped may contain information.

The various embodiments described above are only some of the various embodiments, and various embodiments in which the technical features of the various embodiments of the present disclosure are reflected are detailed descriptions to be described below by those of ordinary skill in the art. It can be derived and understood based on

According to various embodiments, a method and apparatus for analyzing a crack based on artificial intelligence may be provided.

According to various embodiments, a crack inspection of a structure may be efficiently performed.

According to various embodiments, accuracy of crack measurement of a structure may be improved.

The effects obtainable from the various embodiments are not limited to the above-mentioned effects, and other effects not mentioned are clearly derived to those of ordinary skill in the art based on the detailed description below. can be understood

BRIEF DESCRIPTION OF THE DRAWINGS The accompanying drawings, which are included as part of the detailed description to aid understanding of the various embodiments, provide various embodiments and, together with the detailed description, explain the technical features of the present disclosure.
1 is a view for explaining a crack analysis system in which a method of operating an apparatus for crack analysis according to various embodiments may be implemented.
2 is a diagram illustrating a configuration of a first device and/or a second device according to various embodiments of the present disclosure;
3 is a diagram illustrating an example of a method of operating a user device and a server according to various embodiments of the present disclosure;
4 is a diagram illustrating an example of a method of operating a server according to various embodiments of the present disclosure;
5 is a diagram illustrating an example of correction of a first image according to various embodiments of the present disclosure;
6 is a diagram illustrating an example of a sub-image according to various embodiments of the present disclosure;
7 is a diagram illustrating an example of a sub-image according to various embodiments of the present disclosure;
8 is a diagram illustrating an example of a crack length calculation process according to various embodiments.
9 is a view showing an example of a crack thickness calculation process according to various embodiments.
10 is a diagram illustrating a process of obtaining a preset crack analysis model according to various embodiments of the present disclosure;
11 is a diagram illustrating an operating method of an apparatus for acquiring a preset crack analysis model according to various embodiments of the present disclosure;

The following embodiments combine elements and features of various embodiments in a predetermined form. Each component or feature may be considered optional unless explicitly stated otherwise. Each component or feature may be implemented in a form that is not combined with other components or features. In addition, various embodiments may be configured by combining some components and/or features. The order of operations described in various embodiments may be changed. Some features or features of one embodiment may be included in another embodiment, or may be replaced with corresponding features or features of another embodiment.

In the description of the drawings, procedures or steps that may obscure the gist of various embodiments are not described, and procedures or steps that can be understood at the level of those of ordinary skill in the art are also not described. did.

Throughout the specification, when a part is said to "comprising or including" a certain component, it does not exclude other components unless otherwise stated, meaning that other components may be further included. do. In addition, terms such as "... unit", "... group", and "module" described in the specification mean a unit that processes at least one function or operation, which is hardware or software or a combination of hardware and software. can be implemented as Also, "a or an", "one", "the" and like related terms are used herein in the context of describing various embodiments (especially in the context of the claims that follow). Unless otherwise indicated or clearly contradicted by context, it may be used in a sense including both the singular and the plural.

Hereinafter, preferred embodiments according to various embodiments will be described in detail with reference to the accompanying drawings. DETAILED DESCRIPTION The detailed description set forth below in conjunction with the appended drawings is intended to describe exemplary embodiments of various embodiments, and is not intended to represent the only embodiments.

In addition, specific terms used in various embodiments are provided to help the understanding of various embodiments, and the use of these specific terms may be changed to other forms without departing from the technical spirit of various embodiments. .

1. Implementation of crack analysis system

1 is a view for explaining a crack analysis system in which a method of operating an apparatus for crack analysis according to various embodiments may be implemented.

Referring to FIG. 1 , a crack analysis system according to various embodiments may be implemented in various types of devices. In the description of various embodiments, a crack analysis system according to various embodiments may be implemented by an electronic device. For example, the crack analysis system may be implemented in the first device 100 and/or the second device 200 . In other words, the first device 100 and/or the second device 200 may perform operations according to various embodiments based on a crack analysis system implemented in each device. Meanwhile, the crack analysis system according to various embodiments is not limited to that shown in FIG. 1 , and may be implemented in more various devices and/or servers.

The first device 100 according to various embodiments may perform wireless and/or wired communication with the second devices 100 . For example, the first device may be a robot, a vehicle, a portable device, a drone, a smart phone, a computer, a wearable device, a smart pad, a notebook computer, and the like, but is not limited thereto.

The second device 200 according to various embodiments may be an apparatus that performs wireless and/or wired communication with the first devices 100 and includes a database having a large storage capacity. For example, the second device 200 may interwork with the plurality of first devices 100 . Although not shown, a separate device for controlling/managing the second device may be provided.

A crack analysis system according to various embodiments may include various modules for operation. Modules included in the crack analysis system are configured such that a physical device (eg, the first device 100 and/or the second device 200 ) in which the crack analysis system is implemented (or included in the physical device) performs a specified operation. It may be computer code or one or more instructions embodied to be able to do so. In other words, the physical device in which the crack analysis system is implemented stores a plurality of modules in the form of computer code in the memory, and when the plurality of modules stored in the memory are executed, the plurality of modules are designated by the physical device corresponding to the plurality of modules. actions can be performed.

2 is a diagram illustrating a configuration of a first device and/or a second device according to various embodiments of the present disclosure;

Referring to FIG. 2 , the first device and/or the second device may include an input/output unit 210 , a communication unit 220 , a database 230 , and a processor 240 .

The input/output unit 210 may be various interfaces or connection ports that receive user input or output information to the user. The input/output unit 210 may be divided into an input module and an output module, and the input module receives a user input from a user. The user input may be made in various forms including a key input, a touch input, and a voice input. Examples of input modules that can receive such user input include a traditional keypad, keyboard, and mouse, as well as a touch sensor that detects a user's touch, a microphone that receives a voice signal, a camera that recognizes gestures through image recognition, and the like; A proximity sensor composed of an illuminance sensor or infrared sensor that detects user approach, a motion sensor that recognizes a user's motion through an acceleration sensor or a gyro sensor, and various other input means for sensing or receiving various types of user input It is a comprehensive concept that includes all Here, the touch sensor may be implemented as a piezoelectric or capacitive touch sensor for detecting a touch through a touch panel or a touch film attached to the display panel, an optical touch sensor for detecting a touch by an optical method, and the like. In addition, the input module may be implemented in the form of an input interface (USB port, PS/2 port, etc.) for connecting an external input device that receives a user input instead of a device that detects a user input by itself. In addition, the output module can output various information and provide it to the user. The output module is a comprehensive concept including a display that outputs an image, a speaker that outputs a sound, a haptic device that generates vibration, and other various types of output means. In addition, the output module may be implemented in the form of a port-type output interface for connecting the above-described individual output means.

For example, the display-type output module may display text, still images, and moving images. The display includes a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a flat panel display (FPD), and a transparent display. display), a curved display, a flexible display, a three-dimensional display, a holographic display, a projector, and various types of devices capable of performing other image output functions. It is a concept meaning an image display device in a broad sense that includes all of them. Such a display may be in the form of a touch display integrally formed with the touch sensor of the input module.

The communication unit 220 may communicate with an external device. Accordingly, the first device and/or the second device may transmit/receive information to and from an external device through the communication unit. For example, the first device and/or the second device may communicate with an external device so that information stored and generated in the crack analysis system is shared using a communication unit.

Here, communication, that is, transmission and reception of data may be performed by wire or wirelessly. To this end, the communication unit includes a wired communication module that accesses the Internet through a local area network (LAN), a mobile communication module that accesses a mobile communication network through a mobile communication base station and transmits and receives data, and a wireless local area network (WLAN) such as Wi-Fi. A short-distance communication module using an area network communication method or a wireless personal area network (WPAN) communication method such as Bluetooth or Zigbee, or a global navigation satellite system (GNSS) such as GPS (Global Positioning System) ) using a satellite communication module or a combination thereof. A wireless communication technology used for communication may include a Narrowband Internet of Things (NB-IoT) for low-power communication. At this time, for example, NB-IoT technology may be an example of LPWAN (Low Power Wide Area Network) technology, and may be implemented in standards such as LTE Cat (category) NB1 and/or LTE Cat NB2, It is not limited. Additionally or alternatively, a wireless communication technology implemented in a wireless device according to various embodiments may perform communication based on LTE-M technology. In this case, as an example, the LTE-M technology may be an example of an LPWAN technology, and may be called by various names such as enhanced machine type communication (eMTC). For example, LTE-M technology is 1) LTE CAT 0, 2) LTE Cat M1, 3) LTE Cat M2, 4) LTE non-BL (non-Bandwidth Limited), 5) LTE-MTC, 6) LTE Machine It may be implemented in at least one of various standards such as Type Communication, and/or 7) LTE M, and is not limited to the above-described name. Additionally or alternatively, the wireless communication technology implemented in the wireless device according to various embodiments may include at least one of ZigBee, Bluetooth, and Low Power Wide Area Network (LPWAN) in consideration of low power communication. may include, but is not limited to the above-mentioned names. For example, the ZigBee technology can create PAN (personal area networks) related to small/low-power digital communication based on various standards such as IEEE 802.15.4, and can be called by various names.

The database 230 may store various types of information. A database can store data temporarily or semi-permanently. For example, in the database, an operating program (OS) for driving the first device and/or the second device, a program or application (eg, a web application) for generating data or Braille for hosting a web site ) may be stored. In addition, the database may store the modules in the form of computer code as described above.

Examples of the database 230 include a hard disk (HDD), a solid state drive (SSD), a flash memory, a read-only memory (ROM), a random access memory (RAM), and the like. This can be. Such a database may be provided in a built-in type or a detachable type.

The processor 240 controls the overall operation of the first device 100 and/or the second device 200 . To this end, the processor 340 may perform calculation and processing of various types of information and may control operations of components of the first device and/or the second device. For example, the processor 340 may execute a program or application for crack analysis. The processor 240 may be implemented as a computer or a similar device according to hardware software or a combination thereof. In hardware, the processor 240 may be provided in the form of an electronic circuit that processes electrical signals to perform a control function, and in software, it may be provided in the form of a program for driving the processor 240 in hardware. Meanwhile, in the following description, unless otherwise specified, the operations of the first device and/or the second device may be interpreted as being performed under the control of the processor 240 . That is, when the modules implemented in the crack analysis system are executed, the modules may be interpreted as controlling the processor 240 to perform the following operations on the first device and/or the second device.

In summary, various embodiments may be implemented through various means. For example, various embodiments may be implemented by hardware, firmware, software, or a combination thereof.

In the case of implementation by hardware, the method according to various embodiments may include one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), FPGAs (field programmable gate arrays), a processor, a controller, a microcontroller, may be implemented by a microprocessor.

In the case of implementation by firmware or software, the method according to various embodiments may be implemented in the form of a module, procedure, or function that performs the functions or operations described below. For example, the software code may be stored in a memory and driven by a processor. The memory may be located inside or outside the processor, and data may be exchanged with the processor by various known means.

2. Operation of the device according to various embodiments

Hereinafter, various embodiments will be described in more detail based on the above technical idea. The various embodiments described below may be combined in whole or in part to constitute other various embodiments unless mutually exclusive, which may be clearly understood by those of ordinary skill in the art.

The contents of Section 1 described above may be applied to various embodiments described below. For example, operations, functions, terms, etc. that are not defined in various embodiments described below may be performed and described based on the contents of the first section.

Unless specifically stated otherwise, in the description of various embodiments, an apparatus and/or an electronic device may be a first device and/or a second device.

Hereinafter, user equipment according to an implementation example of the first device 100 according to various embodiments will be described, but other devices performing similar functions may also be the first device.

Hereinafter, a server according to an implementation example of the second device 200 according to various embodiments will be described, but other devices performing similar functions may also be the second device.

Unless specifically stated otherwise, in the description of various embodiments, A/B/C may mean A and/or B and/or C.

Unless specifically stated otherwise, in the description of various embodiments, more than/above A may be substituted with more than/above A.

Unless specifically stated otherwise, in the description of various embodiments, less than/below B may be replaced with less than/below B.

Unless otherwise specified, information used by the device/information output/displayed by the device in the description of various embodiments is information directly identified/obtained by the device, information stored in a database included in the device, or a server and/or information received from other external devices.

2.1. crack analysis behavior

In the following description, various embodiments have been described on the premise that the server, which has obtained image information from the user device, performs an operation related to crack analysis. However, according to various embodiments, the user device that has acquired image information from the server It is also possible to perform actions related to crack analysis. Alternatively, according to various embodiments, each of one server or one user device may directly obtain image information (eg, directly input from a user) and perform an operation related to crack analysis. Alternatively, according to various embodiments, a plurality of user devices and/or a plurality of servers are provided, and an operation related to image information transmission/reception and crack analysis between the plurality of user devices and/or the plurality of servers is distributed for each operation. and may be implemented separately.

3 is a diagram illustrating an example of a method of operating a user device and a server according to various embodiments of the present disclosure;

4 is a diagram illustrating an example of a method of operating a server according to various embodiments of the present disclosure;

3 and 4 , in operations 301 and 401 according to various embodiments, the user device may transmit information related to a first image, and the server may receive/obtain it.

In operations 303 and 403 according to various embodiments, the server detects one or more cracks included in a specific wall based on output information obtained in response to input information including the first image being input to a preset crack analysis model. It is possible to obtain analysis information including one or more analysis results for the .

For more specific details of the operation of the user device and/or the server according to various embodiments, reference may be made to the following description.

According to various embodiments, the first image may be visual information on a specific wall among a plurality of walls included in the structure. For example, the structure may be, but is not limited to, an architecture, a building, a building, a road, a dam, an embankment, and the like. For example, the first image and/or information related to the first image may include time of flight (TOF) information, which may include various It can be used with a method according to embodiments.

According to various embodiments, when input information is input, the preset crack analysis model may acquire a plurality of sub-images based on the first image for crack analysis. According to various embodiments, a method for obtaining a plurality of sub-images may vary according to a shape of the first image. For example, when the first image is not the image of the front of the wall, a process of correcting the first image may be included.

According to various embodiments, when an angle between a first normal line perpendicular to the surface of the first image and a second normal line perpendicular to a specific wall surface obtained from the first image is less than or equal to a preset threshold, the first image is It can be determined/identified as being a front view of a specific wall surface. In this case, according to various embodiments, the plurality of sub-images may be a first image segmented into a preset size.

According to various embodiments, when the angle between the first normal perpendicular to the surface of the first image and the second normal perpendicular to the specific wall surface obtained from the first image exceeds a preset threshold, the first image is the specific wall surface. It may be determined/identified as not being a front view. In this case, according to various embodiments, the first image may be corrected so that the angle between the first normal and the second normal is equal to or less than a preset threshold according to a preset crack analysis model. According to various embodiments, the corrected first image may be a front view of a specific wall surface. According to various embodiments, the plurality of sub-images may be a corrected first image divided into the preset size.

5 is a diagram illustrating an example of correction of a first image according to various embodiments of the present disclosure;

Referring to FIG. 5 , according to various embodiments, the preset crack analysis model may acquire a first normal perpendicular to an input first image and a second normal perpendicular to a specific wall surface included in the first image. have. In the example of FIG. 5 , since the angle between the first normal and the second normal exceeds a preset threshold, the preset crack analysis model corrects the first image so that the angle between the first and second normal is less than or equal to the preset threshold. As much as possible, the first image may be converted into a front view of a specific wall surface.

6 and 7 are diagrams illustrating an example of a sub-image according to various embodiments.

Referring to FIG. 6 , for example, a specific crack may be included as 'all' in a specific sub-image. For example, the preset crack analysis model may perform analysis of cracks in units of sub-images to obtain analysis results for specific cracks included as a whole.

Referring to FIG. 7 , for example, a specific crack may be included across two or more sub-images. For example, the preset crack analysis model may perform analysis of cracks in units of sub-images to obtain analysis results for each part of a specific crack included in each sub-image. For example, the preset crack analysis model identifies pixels representing cracks located on the edge of each sub-image, and sets the starting pixel (pixel_start) at which a specific crack starts on the edge of each sub-image and Based on concatenating the end pixel (pixel_end), it is possible to obtain an analysis result for a specific crack across two or more sub-images.

According to various embodiments, the preset crack analysis model may distinguish a crack image included in each of a plurality of sub-images and a specific wall image. According to various embodiments, a crack image may be associated with at least some of the one or more cracks. According to various embodiments, at least a portion of a crack and/or a copper crack may be included in the crack image. According to various embodiments, the specific wall image may be associated with at least a portion of the specific wall surface. According to various embodiments, a specific wall surface and/or at least a part of the specific wall surface may be included in the specific wall image. According to various embodiments, the preset crack analysis model may distinguish a crack from a wall other than a crack from the plurality of sub-images, and may perform analysis on the divided crack.

According to various embodiments, the preset crack analysis model may obtain a plurality of crack pixels for at least some of one or more cracks from the crack image.

According to various embodiments, the preset crack analysis model may acquire a plurality of sub-crack pixels for obtaining a length of at least a part of one or more cracks among the plurality of crack pixels.

According to various embodiments, the preset crack analysis model may include a first starting pixel corresponding to a first starting point of one or more cracks and a first starting point corresponding to a first ending point of one or more cracks among the plurality of sub-crack pixels. An end pixel may be identified.

According to various embodiments, a length of at least a portion of one or more cracks may be calculated based on the preset crack analysis model sequentially tracking a plurality of sub-crack pixels from a first start pixel to a first end pixel.

8 is a diagram illustrating an example of a crack length calculation process according to various embodiments.

8 is an example in which a specific crack is included as a whole on a specific sub-image. This is an example, and the same method can be applied even when a part of a specific crack is included on a specific sub-image.

Referring to FIG. 8 , according to various embodiments, a specific crack on a specific sub-image may be represented by a plurality of crack pixels. According to various embodiments, the preset crack analysis model may acquire a plurality of sub-crack pixels for obtaining a specific crack length from among the plurality of crack pixels. Pixels located on the dotted line in FIG. 8 may be a plurality of sub-cracked pixels. For example, after a specific crack image is divided at equal intervals along the longitudinal direction of the specific crack, a set of middle pixels among pixels included in each divided edge may be a plurality of sub-crack pixels, but the plurality of sub-cracks may be The method by which the pixels are obtained is not limited thereto.

According to various embodiments, the preset crack analysis model may identify a start pixel (first start pixel) (pixel_start) and an end pixel (first end pixel) (pixel_end) among the plurality of sub-crack pixels. According to various embodiments, the preset crack analysis model calculates the distance between each pixel while tracking a plurality of sub-crack pixels from the first start pixel to the first end pixel, and calculates the total sum of the distances between the pixels and TOF information. They can be applied together to calculate the length of the crack.

According to various embodiments, the preset crack analysis model uses a specific sub-crack pixel Pn among a plurality of sub-crack pixels as an endpoint and a first sub-crack pixel Pn-1 immediately before the specific sub-crack pixel Pn. ) is the difference between the first vector having a starting point and a second vector having a specific sub-crack pixel (Pn) as a starting point and a second sub-crack pixel (Pn+1) immediately after the specific sub-crack pixel (Pn) as an endpoint, It is possible to obtain a normal vector at a specific sub-crack pixel Pn.

According to various embodiments, the preset crack analysis model may include a second starting pixel corresponding to a second starting point of one or more cracks based on tracking at least some of the plurality of sub-crack pixels along a direction of a normal vector. and a second end pixel corresponding to the first end point of the one or more cracks.

According to various embodiments, the preset crack analysis model may calculate a candidate thickness corresponding to a specific sub-crack pixel Pn based on an interval between the second start pixel and the second end pixel .

According to various embodiments, n is a natural number, and the preset crack analysis model may perform the same process for all pixels on the plurality of sub-crack pixels.

According to various embodiments, the preset crack analysis model may calculate a maximum value among a plurality of candidate thicknesses corresponding to each of the plurality of sub-crack pixels as the thickness of the crack.

9 is a view showing an example of a crack thickness calculation process according to various embodiments.

9 is an example in which a specific crack is included as a whole on a specific sub-image. This is an example, and the same method can be applied even when a part of a specific crack is included on a specific sub-image.

Referring to FIG. 9 , according to various embodiments, a specific crack on a specific sub-image may be represented by a plurality of crack pixels. According to various embodiments, the preset crack analysis model may acquire a plurality of sub-crack pixels for obtaining a specific crack length from among the plurality of crack pixels. The pixels located on the dotted line in FIG. 9 may be a plurality of sub-cracked pixels. For example, after a specific crack image is divided at equal intervals along the longitudinal direction of the specific crack, a set of middle pixels among pixels included in each divided edge may be a plurality of sub-crack pixels, but the plurality of sub-cracks may be The method by which the pixels are obtained is not limited thereto.

According to various embodiments, the preset crack analysis model includes a specific pixel (Pn) among the plurality of sub-crack pixels, and a first sub-crack pixel (Pn-1) immediately before a specific pixel (Pn) on the plurality of sub-crack pixels. , it is possible to identify the second sub-crack pixel (Pn+1) immediately after the specific pixel (Pn) on the plurality of sub-crack pixels.

According to various embodiments, the preset crack analysis model includes a first vector V1 with a first sub-crack pixel Pn-1 as a starting point and a specific pixel Pn as an end point and a specific pixel Pn. A second vector V2 may be obtained with a point and a second sub-crack pixel Pn+1 as an endpoint.

According to various embodiments, the preset crack analysis model may obtain a normal vector (V2-V1) that is a difference between the second vector and the first vector, and calculates a plurality of crack pixels on an extension line along the longitudinal direction of the normal vector. By tracking, a second start pixel (pixel_start) and a second end pixel (pixel_end) on the extension line may be identified.

According to various embodiments, the preset crack analysis model may calculate a candidate thickness that can be a candidate for the thickness of a crack by applying the distance from the second start pixel to the second end pixel and TOF information together.

According to various embodiments, the preset crack analysis model may calculate candidate thicknesses on a plurality of sub-crack pixels, and obtain a maximum value among the calculated candidate thicknesses as the thickness of a specific crack.

According to various embodiments, the information related to the first image may include location information on the location of a specific wall including the first image.

According to various embodiments, the analysis information may include information in which one or more analysis results are mapped with location coordinates corresponding to a specific wall surface identified based on location information in one or more of the second images related to the structure. .

According to various embodiments, the second image may include a plurality of structure images for a plurality of floors included in the structure. For example, the plurality of structural images may be a blueprint of each floor, an architectural drawing, and the like, but is not limited thereto. According to various embodiments, the location information may include information on a floor including a specific wall surface among a plurality of floors. According to various embodiments, a structure image of a layer including a specific wall surface among a plurality of structural images may be identified based on information on a layer including a specific wall surface. According to various embodiments, a structure image of a layer including a specific wall surface, location coordinates corresponding to a specific wall surface, and one or more analysis results may be mapped. According to various embodiments, the location coordinates may be two-dimensional coordinates based on a preset reference coordinate system for the location of a specific wall surface on a structural image of a floor including a specific wall surface.

According to various embodiments, the mapped information and/or the mapping may include displaying/mapped analysis results for cracks on the specific wall surface on the location of the specific wall surface on the structural image including the specific wall surface. .

2.2. Crack analysis model acquisition behavior

An artificial intelligence system is a computer system that implements human-level intelligence, and unlike the existing rule-based smart system, the machine learns, judges, and becomes smarter by itself. The more the artificial intelligence system is used, the higher the recognition rate and the more accurate understanding of user preferences, and the existing rule-based smart systems are gradually being replaced by deep learning-based artificial intelligence systems.

Artificial intelligence technology consists of machine learning (deep learning) and element technologies using machine learning.

Machine learning is an algorithm technology that categorizes/learns the characteristics of input data by itself, and element technology uses machine learning algorithms such as deep learning to simulate functions such as cognition and judgment of the human brain. It consists of technical fields such as understanding, reasoning/prediction, knowledge expression, and motion control.

Various fields where artificial intelligence technology is applied are as follows. Linguistic understanding is a technology for recognizing and applying/processing human language/text, and includes natural language processing, machine translation, dialogue system, question and answer, and speech recognition/synthesis. Visual understanding is a technology for recognizing and processing objects like human vision, and includes object recognition, object tracking, image search, human recognition, scene understanding, spatial understanding, image improvement, and the like. Inferential prediction is a technology for logically reasoning and predicting by judging information, and includes knowledge/probability-based reasoning, optimization prediction, preference-based planning, and recommendation. Knowledge expression is a technology that automatically processes human experience information into knowledge data, and includes knowledge construction (data generation/classification) and knowledge management (data utilization). Motion control is a technology for controlling autonomous driving of a vehicle and movement of a robot, and includes motion control (navigation, collision, driving), manipulation control (action control), and the like.

In the following description, various embodiments have been described on the premise that the server performs the model acquisition operation, but according to various embodiments, another server outside the server may perform the model acquisition operation. Alternatively, according to various embodiments, a plurality of user devices and/or a plurality of servers may be provided so that each operation of the model acquisition operation is distributed among the plurality of user devices and/or the plurality of servers to be separately executed. may be

10 is a diagram illustrating a process of acquiring a preset crack analysis model according to various embodiments of the present disclosure;

11 is a diagram illustrating an operating method of an apparatus for obtaining a preset crack analysis model according to various embodiments of the present disclosure;

Referring to FIG. 10 , according to various embodiments, the server may collect data for learning. According to various embodiments, the training data may include a plurality of wall front images, a plurality of wall non-frontal images, a plurality of wall crack images for wall cracks, and images for candidates that are likely to be confused with wall cracks. can For example, candidates that are likely to be confused with wall cracks may include engraved decorations on the wall and seams on the wall.

According to various embodiments, based on the plurality of front images and the plurality of non-frontal images of the wall, it may be learned to distinguish between a front view of a wall surface and a non-front view of the wall surface. According to various embodiments, based on a plurality of wall crack images for wall cracks and images of candidates that are likely to be confused with wall cracks, it may be learned to increase the accuracy of wall crack identification.

According to various embodiments, the server may acquire training data based on the training data. For example, the training data may be pre-processed/transformed so that the training data can be input to the AI engine for learning. For example, the training data may include data for training and data for testing.

For example, the server may classify the training data into training data and test data. For example, data for training may be used to train a model, and data for testing may be used to validate and update the trained AI engine.

For example, the server may train the AI engine based on the training data. For example, the AI engine may be an artificial intelligence engine that may be learned based on an artificial intelligence algorithm. For example, artificial intelligence engines include Convolutional Neural Network (CNN), Deep Neural Network (DNN), Recurrent Neural Network (RNN), Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), Bidirectional Recurrent Deep Neural Network (BRDNN). ) or Deep Q-Networks, but is not limited thereto. For example, the preset crack analysis model obtained by this learning method may apply RES (linear center object search) and VGG (circular center object search) in crack analysis, but is not limited thereto.

According to various embodiments, the server may acquire processed data for data for testing based on the learned model.

According to various embodiments, the server may obtain feedback information on the processing data.

For example, the feedback information may be input from a server administrator.

As another example, the feedback information, if there is pre-processed learning data for the processing data, the server may compare the pre-processed learning data and the processing data, and generate feedback information based on the comparison result have. For example, the training data may include pre-processing and post-processing versions of the data, and the server processes the pre-processing data based on the learned model to generate processing data, and after processing included in the processing data and the learning data. The versions may be compared and feedback may be obtained based on a result of the comparison.

According to various embodiments, the server may update the trained model based on the feedback.

For example, the server modifies the characteristics of the learned model (eg, the characteristics of CNN) based on the feedback, processes the test data based on the modified model, and obtains the feedback again and updates the process. Can be repeated.

According to various embodiments, when the number of iterations reaches a preset threshold value (eg, 7-9 times), the server may terminate the operation and acquire the learned AI engine as a recommendation model.

Referring to FIG. 11 , according to various embodiments, in operation 1101, the server may train the AI engine based on training data among training data and test data obtained based on processing the training data. .

According to various embodiments, in operation 1103 , the server may acquire feedback information on the processed data output in response to the test data being input to the learned AI engine.

According to various embodiments, in operation 1105 , the server may update the AI engine based on the feedback information.

According to various embodiments, in operation 1107 , the server may compare the number of times operations 1101 to 1105 are repeated with a preset threshold value (N_TH). For example, a count value corresponding to the number of repetitions of operations 1101 to 1105 may be set. For example, the initial value of the count value may be 0. For example, whenever operations 1101 to 1105 are repeated once, the count value may be increased by one. For example, the server may compare this count value with a preset threshold value (N_TH).

According to various embodiments, if the number of repetitions does not match the preset threshold value (ie, if the number of repetitions is less than the preset threshold value), the server returns to operation 1101 and performs operations 1101 to 1105 again. can

According to various embodiments, the server may terminate the process when the number of repetitions matches a preset threshold.

Various embodiments may be embodied in other specific forms without departing from the technical idea and essential characteristics thereof. Accordingly, the above detailed description should not be construed as restrictive in all respects but as exemplary. The scope of the various embodiments should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the various embodiments are included in the scope of the various embodiments. In addition, claims that are not explicitly cited in the claims may be combined to form an embodiment or may be included as a new claim by amendment after filing.

Claims (5)

In an electronic device,
memory; and a processor coupled to the memory. including,
The processor is:
Obtaining information related to a first image of a specific wall among a plurality of walls included in the structure,
Acquire analysis information including one or more analysis results for one or more cracks included in the specific wall based on output information obtained in response to input information including the first image into a preset crack analysis model set to do,
As the input information is input, a plurality of sub-images having a preset size based on the first image are obtained by the preset crack analysis model,
The information related to the first image includes location information on the location of the specific wall,
The analysis information includes information in which location coordinates corresponding to the specific wall surface identified based on the location information in one or more of the second images related to the structure and the one or more analysis results are mapped,
The one or more analysis results include information on the length of each of the one or more cracks and information on the thickness of each of the one or more cracks,
The length of each of the one or more cracks is determined according to the preset crack analysis model:
A crack image and a specific wall image included in each of the plurality of sub-images are distinguished,
the crack image is associated with at least a portion of the one or more cracks;
the specific wall image is associated with at least a portion of the specific wall surface,
a plurality of crack pixels for at least some of the one or more cracks are obtained from the crack image;
a plurality of sub-crack pixels for obtaining a length of at least a portion of the one or more cracks among the plurality of crack pixels are obtained;
a first starting pixel corresponding to a first starting point of the one or more cracks and a first ending pixel corresponding to a first ending point of the one or more cracks are identified among the plurality of sub-crack pixels;
The electronic device is obtained based on calculating the length of at least a portion of the one or more cracks based on the tracking of the plurality of sub-crack pixels sequentially from the first start pixel to the first end pixel. delete The method of claim 1,
The thickness of each of the one or more cracks is determined according to the preset crack analysis model:
obtained as a maximum value among a plurality of candidate thicknesses corresponding to each of the plurality of sub-crack pixels. 4. The method of claim 3,
The plurality of candidate thicknesses are determined according to the preset crack analysis model:
A first vector having a specific sub crack pixel Pn as an end point among the plurality of sub crack pixels and a first sub crack pixel Pn-1 immediately before the specific sub crack pixel Pn as a starting point, and the specific sub crack pixel Pn as a starting point the difference between the second vectors with the crack pixel Pn as the starting point and the second sub crack pixel Pn+1 immediately after the specific sub crack pixel Pn as the endpoint. A normal vector is obtained,
Based on the tracking of at least some of the plurality of sub-crack pixels along the direction of the normal vector, a second start pixel corresponding to a second start point of the one or more cracks and a first end point of the one or more cracks a corresponding second end pixel is identified;
and a candidate thickness corresponding to the specific sub-cracked pixel (Pn) is calculated based on the interval between the second start pixel and the second end pixel, and n is a natural number. 5. The method of claim 4,
The preset crack analysis model is preset based on application of machine learning to an artificial intelligence (AI) engine for obtaining the preset crack analysis model.

Images