KR102376290B1 - System for managing defect based on augument reality - Google Patents

Abstract

The present invention discloses a fault management system, and more specifically, the present invention relates to a fault management system based on augmented reality, which can check and manage faults existing in a building through a smartphone or the like by using augmented reality technology for houses such as villas and apartments. According to an embodiment of the present invention, an application for detecting defects or faults in a house is presented by using a sensor mounted on a smartphone, and through this, information received for each user is stored in a defect management server and connected with a construction company, and thus a general user can more easily receive a housing fault and process a fault repair or interior procedure.

Description

Augmented reality-based defect management system {SYSTEM FOR MANAGING DEFECT BASED ON AUGUMENT REALITY}

The present invention relates to a defect management system, and in particular, to an augmented reality-based defect management system that can check and manage defects existing in a building through a smartphone using augmented reality technology for houses such as villas and apartments. it's about

Conventionally, for houses such as villas and apartments, in order to check whether safety rules are complied with during construction or after completion, and whether there are any defects or defects, experts must visit numerous sites and go through the inspection process, etc. There was a limitation in that processing time was delayed and cost burden occurred.

In addition, as most of the requests for repairs and repairs are made through voice calls, frequent phone calls have a lot of trouble for home maintenance companies, and repair companies are also experiencing a decrease in work concentration by answering calls during work. have difficulties such as

As such requests for house defect inspection are received and managed through voice calls, housing defect repair companies do not have easy follow-up management, and frequent disputes arise.

In addition, due to the nature of reliance on on-site work, the results are not immediately confirmed after repair services such as defect repair and breakdown repair, so housing occupants are required to check the processing results through on-site photos, etc. The reality is that feedback of processing results is delayed.

Registered Patent Publication No. 10-2210614 (Announcement date: 2021.02.03.)

The present invention has been devised to solve the above-mentioned problems, and the present invention provides an estimate from a construction company by inputting interior details or checking defects and defects of a house through an application installed on a smartphone or the like by a general occupant. There is a challenge to receive and to be able to commission the work.

In order to solve the above problems, the augmented reality-based defect management system according to an embodiment of the present invention is installed in a user terminal, the user himself checks one or more defects that occur in the house, and receives the defect cases for the checked defects and one or more applications that request an estimate for repair or interior design, and the application, and transfers the registration and reception information of the received defect repair request for the house to the management office terminal, Alternatively, it may include a defect management server for requesting an estimate regarding defect repair or interior for the corresponding defect to one or more construction company terminals registered in each region in response to a request for interior quotation.

The application, a defect check unit that automatically detects and inserts a mark in the image taken through the camera module mounted on the user terminal, upload one or more defect images to be accepted to the defect management server to receive the defect repair A quote requesting a quote by receiving a list of one or more construction companies registered in the defect management server through the requesting A/S reception unit and entering a search keyword, and uploading a defect repair or interior video to one or more construction companies selected by the user It may include an inquiry unit and a quotation request unit that receives defect repair or interior requests including construction items input from the user and transmits them to the defect management server to be disclosed to one or more construction companies registered in the defect management server. there is.

The defect check unit is configured to detect one or more regions in the image where the color of the point where at least two regions meet is a different color from the periphery and out of a critical range as a defect part, and generate a mark indicating the detected region to add the image to the image. can be inserted.

The defect checker may detect a shaded area generated by bending in each area in the image as a defect site, generate a mark indicating the detected area, and insert it into the image.

The defect check unit may detect an area having a color different from the entire color and a threshold range in the image as a defect part, generate a mark indicating the detected area, and insert it into the image.

The defect checker may detect any one non-parallel vertical line or horizontal line in an area including two or more vertical lines or horizontal lines in the image, generate a mark mapped on the detected area, and map the mark to the image.

The defect check unit, when the user terminal is in close contact with a point of the house, using the measurement values for the X, Y, Z axes measured from the geomagnetic sensor mounted on the user terminal, the vertical or horizontal degree of the point of contact of the house can be calculated and displayed.

The A/S reception unit may provide a UI for receiving an input of address information about a house, one or more house photos, and defect details from a user, and transmit the input information to the defect management server.

The quotation inquiry unit receives a search keyword including a region and a construction type, requests a search from the defect management server, and receives a reply from the defect management server and displays a list of companies including company information of one or more construction companies corresponding to the keyword can do.

The quotation request unit may receive an input of region, type of construction, and construction matter, register it in the defect management server, and receive estimate information on the registered matter from one or more construction companies thereafter.

The defect management server receives user information and company information from an application and a construction company terminal, respectively, performs a membership registration procedure to register users and companies as members, and a member management unit that processes a login procedure when a login request is made, the user information and a database for storing company information, and for storing defect information on received defect cases, an application linkage unit for transmitting and receiving data in conjunction with the application through an information communication network, and defect information for defect cases requested from the application. A defect reception unit that stores and receives and processes the database, a company selection unit that selects one or more among a plurality of construction companies registered with reference to the defect information, and transmits and estimates the defect information to the terminal of the construction company of the selected construction company In accordance with a search request according to a request and a search request according to a search keyword input of the application and a defect processing unit that receives the quotation information from the construction company terminal and responds to the application, extracts and extracts one or more registered construction companies It may include a company search unit that returns the company information to the application.

The defect reception unit receives the region, construction type and construction information from the application and stores it in a database when a quotation is requested, and the company selection unit extracts one or more registered construction companies according to the region, construction type and construction matter And it is possible to request a quotation from the terminal of the construction company of the extracted construction company.

According to an embodiment of the present invention, an application for detecting defects or defects in a house is provided by utilizing a sensor mounted on a smartphone, and through this, information received for each user is stored in a defect management server and connected with a construction company By doing so, general users can more easily receive home defects and handle defect repairs or interior procedures.

1 is a diagram schematically showing the overall configuration of an augmented reality-based defect management system according to an embodiment of the present invention.
2 is a diagram showing the structure of an application of an augmented reality-based defect management system according to an embodiment of the present invention.
3 is a diagram illustrating a structure of a defect management server of an augmented reality-based defect management system according to an embodiment of the present invention.
4 is a diagram illustrating an augmented reality-based defect management method according to an embodiment of the present invention.
5 to 8 are diagrams illustrating examples of screens provided by an application of an augmented reality-based defect management system according to an embodiment of the present invention.
9 is a view showing an example of a screen for an estimate published online by the augmented reality-based defect management system according to an embodiment of the present invention.

Prior to the description, when a part in the entire specification "includes" or "includes" a certain component, it does not exclude other components, but may further include other components unless otherwise stated. means there is In addition, terms such as "...unit", "...server", "...system", etc. described in the specification mean a unit that processes at least one function or operation and may be implemented by hardware, software, or a combination of hardware and software.

In addition, the term "embodiment" in this specification means serving as an illustration, example, or illustration, but the subject matter of the invention is not limited by such examples. Also, "comprising", "comprising", "having" and other similar terms are used, but when used in the claims, " as an open-ended transition word that does not exclude any additional or other element. It is used generically in a manner analogous to the term "comprising".

The various techniques described herein may be implemented in conjunction with hardware or software, or a combination of both, where appropriate. As used herein, terms such as "...Unit", "...Server", "...System" and the like likewise refer to computer-related entities, i.e. hardware , a combination of hardware and software, software or software in execution may be treated as equivalent. In addition, each function implemented in the system of the present invention may be configured as a module unit program, and may be recorded in one physical memory or may be recorded while being distributed between two or more memories and recording media.

In the following description, the term referring to the "augmented reality-based defect management system" of the present invention may be abbreviated as "defect management system" or "system" for convenience of description.

Hereinafter, an augmented reality-based defect management system according to an embodiment of the present invention will be described in detail with reference to the drawings.

1 is a diagram schematically showing the overall configuration of an augmented reality-based defect management system according to an embodiment of the present invention.

1, the augmented reality-based defect management system according to an embodiment of the present invention is installed in the user terminal 50, the user himself checks one or more defects that have occurred in the house, and checks the defect cases for the checked defects. By interworking with one or more applications 100 and applications 100 that receive and request a quote for repair or interior design, register and receive information about defects received in response to a request for repair of defects for the corresponding house, management office terminal Including a defect management server 200 that transmits to 300 and requests an estimate regarding defect repair or interior for the defect case to one or more construction company terminals 400 registered by region in response to a request for defect or interior estimate can do.

The application 100 is an application program installed and executed in the user terminal 50 possessed by the user who wants to use the defect maintenance management service provided by the system of the present invention, and the user checks the home defect by himself/herself and, when a defect occurs, It may request the management server 200 to take action.

In particular, the application 100 according to an embodiment of the present invention uses the camera module 60 mounted on the user terminal 50 to photograph each area of a house to be managed, and to perform a real-time analysis of the captured image. can be automatically identified. In addition, the application 100 may request the defect management server 200 to repair the defect for the identified defective area.

In addition, the application 100 may request various services related to housing defects through interworking with the defect management server 200 .

The aforementioned user terminal 50 includes a Personal Communication System (PCS), a Global System for Mobile communications (GSM), a Personal Digital Cellular (PDC), a Personal Handyphone System (PHS), a Personal Digital Assistant (PDA), and an International Mobile Telecommunication (IMT). )-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminal, smartphone, smartpad, tablet PC (Tablet) All kinds of handheld-based mobile terminal devices such as PC) and various known computing devices such as stationary PCs and notebook computers may be used, but the present invention is not limited thereto.

The defect management server 200 interworks with one or more applications 100 through the information communication network, registers users and construction companies according to the request of the application 100 and the construction company terminal 400 as members, and from the application 100 You can receive the requested defect, extract suitable construction companies for each defect, and connect with the user, or request and deliver an estimate.

In detail, the defect management server 200 of the present invention connects a user and a construction company to each other through an information and communication network for a large-scale housing complex as well as a single house that has been agreed in advance, thereby dealing with the occurrence of defects and repair problems of the house. and can provide various services related to other housing defects to the user.

In addition, the defect management server 200 according to an embodiment of the present invention may further provide a service for connecting a registered construction company and delivering estimate information according to a user's request for interior construction for a house.

The above-mentioned information communication network may be a closed network such as a LAN (Local Area Network) or WAN (Wide Area Network) as an example of a data communication network, as well as a wired telephone network such as a Public Switched Telephone Network (PSTN), An open network such as the Internet may be used. The Internet is a TCP/IP protocol and several services existing in its upper layers, namely HTTP (HyperText Transfer Protocol), Telnet, FTP (File Transfer Protocol), DNS (Domain Name System), SMTP (Simple Mail Transfer Protocol), SNMP ( It may refer to a worldwide open computer network structure that provides Simple Network Management Protocol), NFS (Network File Service), and NIS (Network Information Service). In addition, as an example of a wireless communication network, technical standards or communication methods for mobile communication (eg, Global System for Mobile communication (GSM), long-term evolution (LTE), LTE Advance (LTE-A), CDMA (code division multiple access), WCDMA (wideband CDMA), UMTS (universal mobile telecommunications system), WiBro (Wireless Broadband), or GSM (Global System for Mobile Communications), any one, such as 5G, may be used, but limited to this it is not going to be

According to the above structure, in the augmented reality-based defect management system according to an embodiment of the present invention, users who own or rent houses in each area execute applications using their smartphones, etc. You can easily check the occurrence of defects and handle defect repair procedures.

Hereinafter, the function and structure of the application included in the augmented reality-based defect management system according to an embodiment of the present invention will be described in detail with reference to the drawings.

2 is a diagram showing the structure of an application of an augmented reality-based defect management system according to an embodiment of the present invention. In the following description, the application and each component constituting the same may be written in a known programming language, recorded on a recording medium that can be read and written by a microprocessor, and mounted in a user terminal.

Referring to FIG. 2 , the application 100 of the augmented reality-based defect management system according to an embodiment of the present invention automatically detects a defect in an image captured through a camera module mounted on a user terminal and checks a defect by inserting a mark. The unit 110, the A/S reception unit 120 that uploads one or more defect images to be accepted to the defect management server to request defect repair reception, and one or more construction companies registered in the defect management server through the search keyword input The quotation inquiry unit 130 for requesting an estimate by uploading a defect repair or interior image to one or more construction companies selected by the user, and inputting defect repair or interior requests including construction information input from the user It may include a quotation requesting unit 140 that receives and transmits to the defect management server to be disclosed to one or more construction companies registered in the defect management server.

The defect check unit 110 may detect various defects occurring in the user's house. The application 100 according to an embodiment of the present invention may be installed and executed in a user terminal such as a smartphone, and in particular, a predetermined camera module may be mounted on such a user terminal, or may be connected to at least a separate camera device, and a housing therefrom You can take an image for each area of .

In particular, the defect check unit 110 according to the embodiment of the present invention performs image analysis based on augmented reality technology to automatically check the defect area from the image of the house taken by the camera module, and the defect check unit ( 110) may determine and extract an area in which the color of a background or object that appears in an image captured in real time from the camera module is significantly different from the surrounding area as a defect occurrence area.

In detail, the defect check unit 110 may detect and display a defect existing in the image in various ways according to the form of the defect.

For example, the defect check unit 110 detects one or more regions in the image, in which the color of the point where at least two regions meet, is a different color from the periphery and out of a critical range as a defect part, and displays a mark indicating the detected region. You can create it and embed it in your video.

Alternatively, the defect check unit 110 may detect a shaded area generated by bending in each area as a defective part in the image, generate a mark indicating the detected area, and insert it into the image.

Alternatively, the defect check unit 110 may detect a region having a different color from the entire color in the image as a defect portion, generate a mark indicating the detected region, and insert it into the image.

Alternatively, the defect check unit 110 detects any one non-parallel vertical line or horizontal line in an area including two or more vertical lines or horizontal lines in the image, generates a mark mapped on the detected area, and maps it to the image. can

Alternatively, the defect check unit 110, when the user terminal is in close contact with a point of the house, using the measurement values for the X, Y, and Z axes measured from the geomagnetic sensor mounted on the user terminal, the vertical of the point of contact of the house A degree or a horizontal degree can be calculated and displayed. Accordingly, when the user directly brings the user terminal to the wall, ceiling, floor or the like in the house, the defect check unit 110 may determine the horizontality or the verticality of the part.

As described above, when the defect check unit 110 detects a defect through image analysis, it is determined whether there is a defect through an abrupt color change of the background and the object, thereby minimizing misjudgment and improving the accuracy of the result. In order to increase it, a background and an object appearing in an image may be identified, a defect type may be classified according to the type of the background and an object, and different thresholds may be adaptively set for each defect type.

In detail, a significant number of defects in housing defects can be divided into lifting of the flooring, excessive gaps in door gaps, horizontal/vertical defects, water leakage, lifting, molding breakage, step differences, etc., and by setting different thresholds for each defect type It is possible to increase the accuracy of the defect judgment.

In addition, since the colors constituting most areas are very similar in different types of housing members such as walls or floors, a threshold value is set lower than the average to determine a defect even if a slight color change is detected.

In addition, since the wall, floor, and the corner where the wall and the floor meet, the color of the wall, floor, and corner changes significantly depending on the color and material difference, so the corner may be misjudged as a defect, so multiple colors in one image In this case, a defect can be determined only when a significant color change is detected by setting the threshold higher than the average.

In addition, in order to increase accuracy, when a defect area is displayed on the screen, the user visually checks whether there is an actual defect on the currently displayed screen in real time, and if it is an erroneous judgment by the defect check unit 110 rather than a defect, By capturing and saving the video and reporting it to the defect management server as an exception, the system can use it to adjust the threshold for individual issues in the future.

When a defect is detected by the defect check unit 110 , the A/S reception unit 120 may generate defect information including the image and transmit it to the defect management server to request a defect acceptance for the user's house.

In particular, the A/S reception unit 120 provides a UI for receiving input from the user address information about the house, one or more house photos, and defect details, and transmits the input information to the defect management server, thereby providing a defect management server. You can request to apply for

The quotation inquiry unit 130 may allow the user to check and select a construction company registered in the defect management server by using a search keyword, and request an estimate for repairing defects from one or more selected construction companies. To this end, the quotation inquiry unit 130 may provide a UI for local search and input of the work type, and according to the entered search keyword, a list of one or more construction companies extracted by the defect management server in response to a reply, photos registered by the construction company, and company introduction The contents can be displayed on the screen. The user may select any one construction company included in the company list, and thus the quotation inquiry unit 130 may display the company name, address, homepage, contact information, company photo, etc. for the selected construction company on the screen.

In addition, the quotation inquiry unit 130 may provide a UI for inputting specific construction request contents, and when the user inputs construction request contents and contact information, it is transmitted to the defect management server and the requested contents from the construction company. to be able to check

Here, the quotation inquiry unit 130 may provide not only an estimate for house defect repair, but also a function to search for a construction company and request a quotation for interior construction.

The quotation request unit 140 registers the required defect repair or interior construction details in the defect management server, rather than directly searching and selecting the supplier, thereby providing estimate information for the construction registered from a plurality of construction companies. You can provide a function that allows you to receive .

To this end, the quotation request unit 140 may provide a UI in which the user can input specific details, photos, and contact information for the construction contents requested by the user, and when the above-mentioned contents are input by the user, the request is processed It is transmitted to the defect management server so that it is disclosed to the construction companies registered online, and when the construction company returns the estimate information according to the request, the estimate information is output on the screen to the construction company desired by the user. to select and proceed with construction.

Hereinafter, a defect management server of an augmented reality-based defect management system according to an embodiment of the present invention will be described in detail with reference to the drawings.

3 is a diagram illustrating a structure of a defect management server of an augmented reality-based defect management system according to an embodiment of the present invention.

3, the defect management server 200 of the augmented reality-based defect management system according to an embodiment of the present invention receives user information and company information from the application 100 and the construction company terminal 400, respectively, A member management unit 210 that performs a membership registration procedure to register users and companies as members, processes a login procedure when a login is requested, stores user information and company information, and a database for storing defect information on received defects (220), the application linkage unit 230 for transmitting and receiving data by interworking with the application 100 through the information communication network, and stores the defect information about the defect request from the application 100 in the database 220 and accepts processing Defect reception unit 240, a company selection unit 250 that selects one or more among a plurality of construction companies registered with reference to defect information, and transmits defect information to the construction company terminal 400 of the selected construction company and estimates In response to a search request according to the search keyword input of the request, the defect processing unit 260 and the application 100 to receive the quotation information from the construction company terminal 400 and reply to the corresponding application, the corresponding one or more registered It may include a company search unit 270 that extracts the construction company and returns the company information of the extracted construction company to the application 100 .

The member management unit 210 may perform a member registration procedure according to a request of a general user and a construction company who want to use the service provided by the system of the present invention, and may perform a login procedure for the registered member.

The application 100 accessed through the information and communications network may request user registration by transmitting user information including a user name, account name, and password, and the construction company terminal 400 includes a company name, account name, password, etc. You can request company registration by entering your company information. Accordingly, the member management unit 210 may process the member registration procedure and store user information and company information in the database 220, and then, to the login request using the account name of the application 100 and the construction company terminal 400, Approve the login and use the service afterwards.

The database 220 may store user information and company information input during member registration. In addition, the database 220 may store defect information on the received defect case, estimate information uploaded by a construction company, and the like, and may provide necessary information for each service.

In addition, the database 220 may store various setting information for the operation of the defect management server 200 .

The application interworking unit 230 may transmit/receive various data for service provision by interworking with one or more applications 100 through an information communication network. When the application 100 is executed on the user terminal side, a login procedure is performed and when the login is normally completed, the application linkage unit 230 receives a request to receive a defect from the application 100, or receives from the construction company terminal 400 One estimate information may be returned to the application 100 . In addition, the application linkage unit 230 may transmit/receive various data for processing a search service, a quotation inquiry, a request service, and the like.

The defect reception unit 240 may receive the defect information automatically generated from the application 100 to receive a request for receiving the defect, and after the reception process, the user can receive a quote by providing the defect information to one or more construction company terminals 400 . make it possible

In addition, the defect reception unit 240 receives, in addition to the defect information automatically detected by the application 100, the region, work type and construction information input by the user, and when a quotation is requested, it is registered and stored in the database 220 It is possible to request an estimate by transmitting information such as construction details stored in the terminal 400 of one or more construction companies extracted by the company selection unit 250 to be described later.

The company selection unit 250 may select one or more suitable construction companies according to the received and processed defect cases. Defect information may include automatically checked defect images, user name, housing address, defect details or interior details and requests, etc. One or more suitable construction companies can be selected by comparing requirements such as address and construction type.

When one or more construction companies are selected for the received and processed defect cases, the defect processing unit 260 may request a quote by providing defect information to the corresponding construction company terminal 400 . Thereafter, when the construction company terminal 400 returns the estimate information including the cost, time, construction details, etc., the user of the defect can be identified and the estimate information can be transmitted to the application 100 .

In addition, the defect processing unit 260 may provide a contract and payment function for the defect case approved by the user after the selection of the construction company.

The company search unit 270 searches the database 220 using the search keyword input by the user from the application 100 to extract one or more matching construction companies, and lists the company information to the application 100 can be sent to

Hereinafter, an augmented reality-based defect management method according to an embodiment of the present invention will be described in detail with reference to the drawings.

4 is a diagram illustrating an augmented reality-based defect management method according to an embodiment of the present invention. In the following description, even if there is no separate description, the execution subject of each step is the system of the present invention, the applications constituting the system, the defect management server, and its constituent parts.

4, according to the augmented reality-based defect management method according to an embodiment of the present invention, it is possible to provide services such as automatic defect reception, A/S reception, quotation inquiry using a search keyword, and quotation request, First, in the case of the automatic defect reception service, when a user executes an application and selects a defect check button (S100), each area of the house can be photographed as the camera module of the user terminal is driven (S110).

In addition, during image shooting by the camera module, the application analyzes the background, object, color, line angle, etc. appearing in the image in real time (S120), and can detect the occurrence of a defect (S150), and it is determined that the defect has occurred It is possible to generate a mark for an area and to generate defect information including the image. If no defects are detected by the end of video recording, the procedure is terminated.

And, when a defect is detected (S150), the application may request A/S for the corresponding defect according to the user's input, otherwise the procedure ends (S160). When A/S is requested, the subsequent procedure is the same as the A/S reception service.

In addition, according to an embodiment of the present invention, in the automatic defect check service, the application may further provide a leveler function using a geomagnetic sensor mounted on the user terminal, and according to the user's selection, the application drives the leveler (S130) , when the user directly brings the user terminal to the wall, ceiling, or floor in the house, it is possible to determine the horizontality or the verticality in that part (S140).

Thereafter, it is determined whether a defect is detected according to the horizontal or vertical level (S150), and the A/S request step (S160) may be performed or the procedure may be terminated.

In addition, according to an embodiment of the present invention, the system may provide an A/S reception service. When the A/S reception button is selected by the user (S200), the application uploads the defect image stored in the user terminal on the application to generate defect information (S210), and accesses the defect management server through the information and communication network (S220). ).

Next, the application transmits the defect information to the defect management server, requests to receive a defect case, and the defect management server receives and processes the defect repair request (S230), the defect management server refers to the defect information and registers multiple Selects one or more among the construction companies of (S240), and transmits defect information to the terminal of the construction company and requests a reply to the quotation (S250).

Thereafter, when the construction company terminal returns the estimate information, the defect management server transmits the estimate information to the corresponding application.

In addition, according to an embodiment of the present invention, the system may provide a quote inquiry service using a search keyword.

If the defect reception period has passed, or the user who wants to repair the defects of the house individually without going through the management office, or wants to proceed with the interior separately from the defect, selects the search and reception button according to the application progress (S300), the application connects to the defect management server (S310).

Next, when the user selects a desired area, type of construction, etc., and inputs a search keyword (S320), the defect management server extracts one or more matching construction companies based on the search keyword (S330), and the A company list including company information is generated (S340).

Next, the defect management server transmits the list of companies to the application (S350).

Further, according to an embodiment of the present invention, the system may provide a quotation request service. This quotation request service can be used when a user discloses his or her defect or interior matter online through the system, receives quotation information from a plurality of construction companies, and selects a construction company that offers suitable conditions among them.

When the user selects the request for quotation button on the application (S400), the application may receive a pre-photographed defect image or interior image, and contents related to construction matters (S410), and the application transmits the input information to the defect management server It is possible to register the quotation by transmitting (S420).

The registered quotation is disclosed only to a plurality of construction companies registered as members by the defect management server, and the construction company can access the defect management server through the construction company terminal to check the construction details of the published quotation (S430).

Next, the construction company may check the construction details with respect to the estimate case, and the construction company terminal may calculate the estimate and generate the estimate information (S440) and transmit it to the defect management server (S450).

Thereafter, the defect management server may collect estimate information from each construction company and transmit it to an application, and may process a contract and a payment procedure according to the user's approval.

Hereinafter, the technical idea of the present invention will be described in detail through an example of a screen provided by an application of an augmented reality-based defect management system according to an embodiment of the present invention.

5 to 8B are diagrams illustrating examples of screens provided by an application of an augmented reality-based defect management system according to an embodiment of the present invention.

5, the application of the augmented reality-based defect management system according to an embodiment of the present invention is executed in the user terminal 50 such as a smartphone, and the "AR mode defect check" button ( W110), an “A/S reception” button W120, a “Search and Quote inquiry” button W130, and a “Request for quotation” button W140 may be provided.

The "AR mode defect check" button W110 may execute a function of automatically checking a defective area in each area of the house using a camera module or a geomagnetic sensor mounted on the user terminal.

The "A/S reception" button W120 can execute a function of processing a defect case by automatically checking or generating defect information according to a defect or construction content input by the user, and providing it to the defect management server.

The "search and quote inquiry" button W130 may execute a function of searching for a plurality of construction companies registered in the system through the search keyword entered by the user, and providing a list of companies according to the search results.

The "Request Quote" button W140 may provide a function for the user to enter construction details for repair or interior construction, register and disclose them in the system, and receive quotation information from construction companies who have confirmed this.

6A to 6D, an example of an automatic defect check function executed according to the selection of the "AR mode defect check" button W110 provided by the application of the augmented reality-based defect management system according to an embodiment of the present invention As a result, the captured image (img) is displayed on the user terminal in real time according to the operation of the camera module, and the application can automatically determine whether a defect has occurred by analyzing the background, object, line and its color appearing in the image. .

As shown in FIG. 6A , in such a captured image img, the application determines that an area in which a color difference from the surrounding area occurs in the image is a defect, maps a mark PP on the defective area, and displays it together. Also, a save button W102 for saving the current image as a still image, that is, a snapshot, may be displayed on the captured image img.

6B illustrates another type of defect detected by the application of the present invention, and as shown in FIG. 6B , defects such as crying or lifting may occur in the wallpaper or floor covering of the house, which is caused by crying and lifting. It is possible to determine whether a defect exists on the basis of the shading that occurs in the .

6c illustrates another type of defect detected by the application of the present invention, and as shown in FIG. 6c , a defect may occur in the door frame and the like that do not match the verticality of the floor, and the application is in the captured image img. When the angle between the line included in the displayed floor and the line corresponding to the door frame deviates from 90° by more than a threshold, it is determined as a defect, and the display line LL can be overlapped and displayed on the captured image img.

6C illustrates a horizontal ruler or a vertical ruler for measuring defects provided by the application of the present invention. Such a leveler (aa) utilizes a geomagnetic sensor mounted on the user terminal, and when the user executes the leveler of the application and brings the user terminal to the wall or floor of the house to be measured, the measurement is measured by the geomagnetic sensor. Horizontal and vertical degrees are displayed, and the application can determine that a defect has occurred when the measurement result is out of a threshold.

Referring to FIG. 7 , as an example of a reception function executed according to the selection of the "A/S reception" button W120 provided by the application of the augmented reality-based defect management system according to an embodiment of the present invention, the user can use the application After automatically checking the occurrence of a defect through , or when a defect is directly confirmed, the defect may be received through the A/S reception screen W121.

The A/S reception screen W121 includes an attachment window W124 for uploading one or more images stored in the user terminal 50, that is, a photo, an input window W126 through which the user can directly input defect contents as text, and a photo. When the input of the content and the defect content is completed, a transmit button W128 for generating defect information including the input content and transmitting it to the defect management server may be included.

Referring to FIGS. 8A and 8B , the search function executed according to the selection of the "Search and Quote" button W130 provided by the application of the augmented reality-based defect management system according to an embodiment of the present invention is exemplified, The user can enter a search keyword through the search screen W131 to search for a construction company registered in the system and check company information.

This search screen (W131) provides the user with a search window (W132) for searching and selecting the area and construction type of a house, a company photo for one or more construction companies extracted according to a keyword search from a defect management server, and a brief introduction of the company. A company list screen (W124) comprising company information including, when the user selects company information for any one construction company, a company button (W138) that switches to the detailed information screen including the specific information input by the construction company may include

The above-described detailed information screen W139 may include detailed information of the company, a company website address, contact information, photos and portfolios related to the company. In particular, in the embodiment of the present invention, in addition to the company-related contents on the detailed information screen W139, an input window in which the user who has confirmed the detailed information directly inputs the construction request contents and contact information as text on the same screen, and an estimate for the construction company It may further include an input button for selecting a request, contract, and payment function.

9 is a view showing an example of a screen for an estimate published online by the augmented reality-based defect management system according to an embodiment of the present invention.

Referring to FIG. 9 , according to the request of the application of the augmented reality-based defect management system according to the embodiment of the present invention, the screen for the estimate registered in the defect management server and published online is exemplified, and the construction company side construction When the company terminal 400 accesses the defect management server, it is possible to check the quotation screen W401 registered by the user.

In this estimate screen (W401), the information window (W401) containing the area and construction type of the house, which is the construction target set by the user, and the photo of the interior area of the house taken by the user, and the construction entered as text by the user It may include a matter window (W403).

In addition, the quotation screen (W401) may include a quotation button (W408) that can be switched to an input screen for inputting an estimate for the quotation by the construction company that has confirmed the construction details.

The input screen W410 may specifically display construction details input by the user. The input screen W410 may include a detailed information window W411 including detailed information on a user request input as user text and a photo and contact information for a defect repair or interior area uploaded by the user, and construction After the company completes the confirmation of the corresponding content, it may include a quotation sending button (W481) for transmitting the quotation information including the prepared quotation content to the user side.

Accordingly, the user can receive estimate information from a plurality of construction companies at the same time by registering and disclosing defect repair or interior construction details for a house for which the user wants to receive an estimate in the system.

Although many matters are specifically described in the above description, these should be construed as examples of preferred embodiments rather than limiting the scope of the invention. Accordingly, the invention should not be defined by the described embodiments, but should be defined by the claims and equivalents to the claims.

50: user terminal 60: camera module
100: application 110: defect check unit
120: A/S reception department 130: quotation inquiry department
140: quotation request unit 200: defect management server
210: member management unit 220: database
230: application linkage unit 240: defect reception unit
250: company selection unit 260: defect handling unit
270: company search unit 300: management office terminal
400: construction company terminal

Claims (12)

One or more applications that are installed in the user terminal, check one or more defects that occur in the user's own house, receive a defect case for the checked defect, and request a quote for repair or interior defects; and
In conjunction with the application, the registration and reception information of the defect case received in response to the defect repair request for the house is transferred to the management office terminal, and corresponds to one or more construction company terminals registered by region according to the defect or interior quotation request Defect management server that requests quotation regarding defect repair or interior
including,
The application is
A defect check unit that automatically detects a defect in the image taken through the camera module mounted on the user terminal and inserts a mark;
an A/S reception unit that uploads one or more defect images to be accepted to the defect management server to request a defect repair;
a quotation inquiry unit that receives a list of one or more construction companies registered in the defect management server through a search keyword input, and requests a quotation by uploading a defect repair or interior image to one or more construction companies selected by a user; and
A quotation request unit that receives defect repair or interior requests including construction details input from the user and transmits them to the defect management server to be disclosed to one or more construction companies registered in the defect management server
including,
The defect check unit,
In the image, one or more regions in which the color of the point where at least two regions meet is a different color from the periphery and out of the critical range is detected as a defective part, and a mark indicating the detected region is generated and inserted into the image,
In the image, a shaded area generated by bending in each area is detected as a defective part, a mark indicating the detected area is generated and inserted into the image,
In the image, a region having a color different from the entire color range is detected as a defective part, and a mark indicating the detected region is generated and inserted into the image;
In an image, any one non-parallel vertical line or horizontal line is detected in an area including two or more vertical lines or horizontal lines, and a mark mapped on the detected area is generated and mapped to the image;
When the user terminal is in close contact with a point of the house, the vertical or horizontal degree of the point of contact of the house is calculated and displayed using the measurement values for the X, Y, and Z axes measured from the geomagnetic sensor mounted on the user terminal, and ,
Identifies a background and an object appearing in the image, classifies a defect type according to the type of the background and an object, and adaptively sets different thresholds for each defect type,
The user visually checks whether there is an actual defect on the screen on which the defect area is displayed, and if it is an erroneous judgment by the defect check unit rather than a defect, captures and stores the image and reports it as an exception to the defect management server,
The defect management server,
A member management unit that receives user information and company information from the application and construction company terminals, respectively, performs a membership registration procedure to register users and companies as members, and processes a login procedure when a login request is made;
a database for storing the user information and the company information, and storing defect information on the received defect;
an application interworking unit for transmitting and receiving data by interworking with the application through an information communication network;
a defect reception unit for storing and accepting and processing defect information on the defect case requested from the application in the database;
a company selection unit for selecting at least one of a plurality of registered construction companies with reference to the defect information;
Defect processing unit that transmits defect information to the terminal of the construction company of the selected construction company and requests an estimate, and receives the estimate information from the terminal of the construction company and returns to the application; and
In response to a search request according to the search keyword input of the application, a company search unit that extracts one or more registered construction companies corresponding to and returns the company information of the extracted construction companies to the application
includes,
The defect handling unit
An augmented reality-based defect management system that provides contract and payment functions for defects approved by the user after selection of a construction company. delete delete delete delete delete delete The method of claim 1,
The A/S reception unit,
An augmented reality-based defect management system that provides a UI for receiving input from a user of address information about a house, one or more house photos, and defect details, and transmits the inputted information to the defect management server. The method of claim 1,
The quotation inquiry department,
Reinforced, which receives a search keyword including a region and a construction type, requests a search from the defect management server, and receives and displays a list of companies including company information of one or more construction companies corresponding to the keyword from the defect management server Reality-based defect management system. The method of claim 1,
The quotation request unit,
An augmented reality-based defect management system that receives region, type of construction, and construction details, registers it in the defect management server, and then receives estimate information on the registered matters from one or more construction companies. delete The method of claim 1,
The defect reception unit receives input from the application and stores it in a database when a quotation is requested,
The company selection department
Augmented reality-based defect management system that extracts one or more registered construction companies according to the region, type of construction, and construction matters and requests an estimate from the construction company terminal of the extracted construction company.

Images