KR102658689B1 - Reparing method and apparatus based augmented rality for air conditioner - Google Patents

Abstract

A method of displaying air conditioner information in a terminal according to an embodiment of the present invention includes the steps of acquiring an image including an air conditioner from a camera of the terminal, receiving operation history (historical) data of parts of the air conditioner, and obtaining: It may include displaying the image on the display of the terminal and matching operation history data to the image including the air conditioner based on augmented reality and displaying it on the display. The learning model for recognizing parts or determining failure of the present disclosure may include a deep neural network generated through machine learning, and transmission of augmented reality data may be performed in an Internet of Things environment using a 5G network.

Description

Maintenance method and device for air conditioner based on augmented reality {REPARING METHOD AND APPARATUS BASED AUGMENTED RALITY FOR AIR CONDITIONER}

This specification relates to a method and device for maintaining an air conditioner based on augmented reality. More specifically, it relates to a technology for displaying parts of an air conditioner and operation history data of the parts in augmented reality.

In general, an air conditioner (hereinafter referred to as an air conditioner) plays the role of raising or lowering the indoor temperature.

The heat medium circulates through a designated path inside the air conditioner, and the circulating heat medium emits or absorbs heat energy. In an air conditioner, the heat medium is a means of transferring heat energy, and performs heating through the emission of heat energy and cooling through the absorption of heat energy. Additionally, air conditioners also perform air purification, humidification, and dehumidification functions.

An air conditioner is a cooling/heating system that cools or heats the room by repeatedly sucking in indoor air, exchanging heat with a low-temperature refrigerant, and then discharging it into the room. Compressor-condenser-expansion valve-evaporator It is a device that forms a series of cycles consisting of.

Air conditioners are largely divided into integrated and separate types. The separate type can be configured by installing an indoor heat exchanger in the indoor unit and installing an outdoor heat exchanger and compressor in the outdoor unit, and connecting the two separate devices with a refrigerant pipe.

The separate air conditioner may be configured as a multi-type air conditioner in which at least one outdoor unit and a plurality of indoor units connected to the outdoor unit are connected in series.

Recently, as the performance of air conditioners has improved and air conditioners with complex structures have become more common, the types and types of failures that can occur are becoming more diverse, and it is becoming more important for maintenance technicians to quickly determine the reason for the failure of the air conditioner and respond accurately during on-site maintenance. there is.

Technology for performing maintenance based on conventional augmented reality is used in various fields.

As a related technology, Republic of Korea Patent No. 1354133 (hereinafter referred to as 'Related Technology 1') discloses 'remote management type ship ballast water treatment device using augmented reality'.

The treatment device disclosed in Related Art 1 displays status information of each component of the ballast water treatment device using an augmented reality method, and receives and displays operation and failure information of the ballast water treatment device from sensors installed in the ballast water treatment device. A display technology is being disclosed. However, Related Technology 1 has a problem that it is not suitable for application to maintenance of air conditioners because it is limited to receiving alarm information from sensors related to problematic parts.

As another related technology, Republic of Korea Patent Publication No. 10-2015-0084551 (hereinafter referred to as 'Related Technology 2') discloses 'Air conditioner and its failure diagnosis method'.

Related Technology 2 simply discloses a technology for classifying and diagnosing refrigerant leakage and clogging defects based on the detected temperature and detected current of the indoor unit and outdoor unit. Related Technology 2 discloses a technology for diagnosing only refrigerant leakage and clogging defects, and has the problem of not providing a method of conveniently and accurately identifying various causes of failure according to the increasingly complex structure of air conditioners.

The above-mentioned related technologies are technical information that the inventor possessed for the disclosed content or acquired in the process of deriving the disclosed content, and cannot necessarily be said to be known technologies disclosed to the general public before the application for the disclosed content.

One object of an embodiment of the present disclosure is to provide a method for easily determining the cause of failure of parts of an air conditioner by displaying historical data of parts of an air conditioner in augmented reality.

In addition, an embodiment of the present disclosure aims to provide a method for accurately determining the cause of a component's failure by displaying operation history data at the time of failure of an air conditioner component in augmented reality.

In addition, an embodiment of the present disclosure aims to provide a method for accurately determining the cause of failure of parts by displaying the change state of operation history data of parts of an air conditioner in a time-series.

In addition, an embodiment of the present disclosure aims to provide a method for quickly performing maintenance on parts of an air conditioner by displaying the parts locations in augmented reality in the order of the maintenance guide.

One embodiment of the present disclosure is air conditioner information in a terminal that displays historical data of parts of the air conditioner received from a server in augmented reality so that a mechanic can easily and accurately determine the cause of the failure of the air conditioner. A display method and device can be provided.

A method of displaying air conditioner information according to an embodiment of the present disclosure includes the steps of acquiring an image including an air conditioner from a camera of a terminal, receiving operation history data of parts of the air conditioner, and the acquired image. It may include displaying on the display of the terminal and matching operation history data to the image including the air conditioner based on augmented reality and displaying on the display.

An augmented reality display device according to an embodiment of the present disclosure includes a camera, a display, a processor, and a memory that is electrically connected to the processor and stores at least one code executed by the processor, and the memory includes a processor. When executed, the processor controls the camera to recognize the parts of the air conditioner in the acquired image, receives the operation history data of the parts, and applies the operation history data to the image including the air conditioner based on augmented reality. You can store code that causes the image to be matched and displayed on the display along with the image.

The augmented reality-based air conditioner maintenance method and device according to an embodiment of the present disclosure allows a maintenance person to easily and accurately determine the cause of a failure of an air conditioner.

In addition, the air conditioner maintenance method and device according to an embodiment of the present disclosure identifies not only the current operation history data of the part, but also the change in operation history data from the time of failure in chronological order, and determines the cause of the failure of the air conditioner. can be judged accurately.

In addition, the air conditioner maintenance method and device according to an embodiment of the present disclosure is based on an interface that is convenient for checking operation history data of parts, so that a maintenance person can easily and accurately determine the cause of failure of the air conditioner.

In addition, the air conditioner maintenance method and device according to an embodiment of the present disclosure displays the location of the parts according to the parts maintenance guide based on augmented reality, so that the maintenance person can easily and quickly repair the air conditioner for failure of various air conditioners. It can be maintained.

1 is an exemplary diagram of an environment for performing a method of maintaining an air conditioner according to an embodiment of the present invention.
Figure 2 is a flowchart for explaining a method of maintaining an air conditioner according to an embodiment of the present invention.
Figure 3 is an exemplary diagram of performing a maintenance method for an air conditioner according to an embodiment of the present invention.
4 to 6 are exemplary diagrams of the interface of an air conditioner maintenance device according to an embodiment of the present invention.

Hereinafter, embodiments disclosed in the present specification will be described in detail with reference to the attached drawings. However, identical or similar components will be assigned the same reference numbers regardless of reference numerals, and duplicate descriptions thereof will be omitted. The suffixes “module” and “part” for components used in the following description are given or used interchangeably only for the ease of preparing the specification, and do not have distinct meanings or roles in themselves. Additionally, in describing the embodiments disclosed in this specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in this specification, the detailed descriptions will be omitted. In addition, the attached drawings are only for easy understanding of the embodiments disclosed in this specification, and the technical idea disclosed in this specification is not limited by the attached drawings, and all changes included in the spirit and technical scope of the present invention are not limited. , should be understood to include equivalents or substitutes.

Terms containing ordinal numbers, such as first, second, etc., may be used to describe various components, but the components are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is said to be “connected” or “connected” to another component, it is understood that it may be directly connected or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

1 is an exemplary diagram of an environment in which a maintenance method for an air conditioner (hereinafter referred to as an air conditioner) is performed and an air conditioner maintenance device operates in accordance with an embodiment of the present disclosure.

The environment for performing the air conditioner maintenance method according to an embodiment of the present disclosure may include an air conditioner 300, a terminal 100 operating as an augmented reality display device, and an air conditioner monitoring server 200.

The air conditioner 300 may include an outdoor unit 320 that includes a compressor and an outdoor fan motor, and an indoor unit 310 that is electrically connected to the outdoor unit and performs air conditioning in the air conditioning area.

The terminal 100 includes a camera, and can display a front image captured by the camera including an air conditioner on a display. The terminal may receive operation history data of parts of the air conditioner included in the captured image from the air conditioner monitoring server 200 or the air conditioner 300 and display it on the display using an augmented reality method. The terminal 100 may be a smartphone supporting augmented reality, a tablet PC, a smart glass terminal including a camera and a small display device, or a head mounted display (HMD).

The air conditioner monitoring server 200 may receive and store operation data related to the operation of parts included in the air conditioner 300 from the air conditioner 300 . In one embodiment, the air conditioner monitoring server 200 determines whether the air conditioner 300 or parts of the air conditioner 300 are abnormal based on the operation data of the parts of the air conditioner 300, or receives a status diagnosis message from the air conditioner 300. can do. The air conditioner monitoring server 200 may transmit operation history data of parts of the air conditioner 300 based on the unique information (ID) of the air conditioner 300 received from the terminal 100 or the location information of the air conditioner 300.

In one embodiment, the air conditioner monitoring server 200 generates a machine learning-based artificial intelligence learning model capable of recognizing the parts of the air conditioner 300 necessary to display the operation history data of the parts in an augmented reality method to the terminal 100. can be transmitted to Whenever a new air conditioner 300 is released, the air conditioner monitoring server 200 may update a learning model capable of recognizing parts of the air conditioner 300 and transmit it back to the terminal 100.

The term 'machine learning' can be used interchangeably with the term 'machine learning'.

A learning model for recognizing parts received by the terminal 100 from the air conditioner monitoring server 200 may include an artificial neural network (ANN).

Artificial neural network is an information processing system that models the operating principles of biological neurons and the connection relationships between neurons. It is an information processing system in which multiple neurons, called nodes or processing elements, are connected in the form of a layer structure.

Artificial neural network is a model used in machine learning. It is a statistical learning algorithm inspired by biological neural networks (especially the brain in the central nervous system of animals) in machine learning and cognitive science.

Specifically, an artificial neural network can refer to an overall model in which artificial neurons (nodes), which form a network through the combination of synapses, change the strength of the synapse connection through learning and have problem-solving capabilities.

The term artificial neural network may be used interchangeably with the term neural network.

An artificial neural network may include multiple layers, and each layer may include multiple neurons. Additionally, artificial neural networks may include synapses that connect neurons.

Artificial neural networks generally use the following three factors: (1) connection patterns between neurons in different layers, (2) learning process to update connection weights, and (3) output values from the weighted sum of inputs received from the previous layer. It can be defined by the activation function it creates.

Artificial neural networks may include network models such as Deep Neural Network (DNN), Recurrent Neural Network (RNN), Bidirectional Recurrent Deep Neural Network (BRDNN), Multilayer Perceptron (MLP), and Convolutional Neural Network (CNN). , but is not limited to this.

In this specification, the term 'layer' may be used interchangeably with the term 'layer'.

Artificial neural networks are divided into single-layer neural networks and multi-layer neural networks depending on the number of layers.

A typical single-layer neural network consists of an input layer and an output layer.

Additionally, a typical multi-layer neural network consists of an input layer, one or more hidden layers, and an output layer.

The input layer is a layer that receives external data. The number of neurons in the input layer is the same as the number of input variables. The hidden layer is located between the input layer and the output layer and receives signals from the input layer, extracts the characteristics, and passes them to the output layer. do. The output layer receives a signal from the hidden layer and outputs an output value based on the received signal. The input signal between neurons is multiplied by each connection strength (weight) and then added. If this sum is greater than the neuron's threshold, the neuron is activated and outputs the output value obtained through the activation function.

Meanwhile, a deep neural network that includes multiple hidden layers between the input layer and the output layer may be a representative artificial neural network that implements deep learning, a type of machine learning technology.

Meanwhile, the term 'deep learning' can be used interchangeably with the term 'deep learning'.

An artificial neural network can be trained using training data. Here, learning refers to the process of determining the parameters of an artificial neural network using learning data to achieve the purpose of classifying, regression, or clustering input data. You can. Representative examples of artificial neural network parameters include weights assigned to synapses or biases applied to neurons.

An artificial neural network learned from training data can classify or cluster input data according to patterns in the input data.

Meanwhile, an artificial neural network learned using training data may be referred to as a trained model in this specification.

Next, the learning method of the artificial neural network is explained.

Learning methods of artificial neural networks can be broadly classified into supervised learning, unsupervised learning, semi-supervised learning, and reinforcement learning.

Supervised learning is a method of machine learning to infer a function from training data.

Among the functions inferred in this way, outputting continuous values can be called regression, and predicting and outputting the class of the input vector can be called classification.

In supervised learning, an artificial neural network is trained given labels for training data.

Here, the label may mean the correct answer (or result value) that the artificial neural network must infer when training data is input to the artificial neural network.

In this specification, when training data is input, the correct answer (or result value) that the artificial neural network must infer is called a label or labeling data.

Additionally, in this specification, setting labels on training data for learning an artificial neural network is referred to as labeling training data.

In this case, the training data and the label corresponding to the training data constitute one training set, and can be input to the artificial neural network in the form of a training set.

Meanwhile, training data represents a plurality of features, and labeling the training data may mean that the features represented by the training data are labeled. In this case, the training data may represent the characteristics of the input object in vector form.

An artificial neural network can use training data and labeling data to infer a function for the correlation between training data and labeling data. And, the parameters of the artificial neural network can be determined (optimized) through evaluation of the function inferred from the artificial neural network.

The structure of an artificial neural network is specified by the model composition, activation function, loss function or cost function, learning algorithm, optimization algorithm, etc., and hyperparameters are set in advance before learning. It is set, and later, through learning, model parameters are set and the content can be specified.

For example, elements that determine the structure of an artificial neural network may include the number of hidden layers, the number of hidden nodes included in each hidden layer, an input feature vector, and a target feature vector.

The terminal 100 can perform the function of a voice agent. A voice agent may be a program that recognizes the user's voice and outputs a response appropriate for the recognized user's voice as a voice.

The terminal 100 may include a wireless communication unit, an input unit, an output unit, an interface unit including a display, memory, a processor, and a power supply unit.

A trained model may be mounted on the terminal 100.

Meanwhile, the learning model may be implemented in hardware, software, or a combination of hardware and software, and when part or all of the learning model is implemented in software, one or more instructions constituting the learning model may be stored in memory.

The wireless communication unit may include at least one of a broadcast reception module, a mobile communication module, a wireless Internet module, a short-range communication module, and a location information module.

The mobile communication module is a technical standard or communication method for mobile communication (e.g., GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), CDMA2000 (Code Division Multi Access 2000), EV-DO ( Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), Long Term (LTE-A) Wireless signals are transmitted and received with at least one of a base station, an external terminal, and a server on a mobile communication network built according to (Evolution-Advanced), etc.).

The wireless Internet module refers to a module for wireless Internet access and may be built into or external to the terminal 100. The wireless Internet module is configured to transmit and receive wireless signals in a communication network according to wireless Internet technologies.

Wireless Internet technologies include, for example, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Wi-Fi (Wireless Fidelity) Direct, DLNA (Digital Living Network Alliance), WiBro (Wireless Broadband), and WiMAX (Worldwide). These include Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), and LTE-A (Long Term Evolution-Advanced).

The short-range communication module is for short-range communication and includes Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, and NFC (Near Field). Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technology can be used to support short-distance communication.

The location information module is a module for acquiring the location (or current location) of a mobile terminal, and representative examples thereof include a Global Positioning System (GPS) module or a Wireless Fidelity (WiFi) module. For example, if the terminal utilizes a GPS module, the location of the mobile terminal can be acquired using signals sent from GPS satellites.

The input unit may include a camera for capturing a front image for augmented reality, a microphone for receiving an audio signal, and a user input unit for receiving information from the user.

In this specification, an artificial neural network whose parameters are determined by learning using training data may be referred to as a learning model or a trained model.

At this time, the learning model can be used to recognize and track parts in an image containing the air conditioner 300 for augmented reality.

The output unit is for generating output related to vision, hearing, or tactile sensation, and may include a display unit.

The display unit displays (outputs) the information processed in the terminal 100, and in particular, displays the image containing the air conditioner 300 captured by the camera unit on the display in real time, and the processor is used in the image containing the air conditioner 300. By applying the learning model, the operation history data of the corresponding part received from the air conditioner monitoring server 200 can be displayed near the recognized part.

The memory stores data that supports various functions of the terminal 100.

The memory includes a plurality of application programs (application programs or applications) running on the terminal 100, data for the operation of the terminal 100, instructions, and data for the operation of the processor (e.g., machine At least one algorithm information for learning, etc.) can be stored.

The memory may store the model learned in the air conditioner monitoring server 200.

At this time, the memory 170 can store the learned model by dividing it into a plurality of versions depending on the learning time or learning progress, if necessary.

The air conditioner monitoring server 200 may operate as a learning device to recognize parts of the air conditioner 300.

The air conditioner monitoring server 200 may transmit a model learned through machine learning or deep learning to the terminal 100 periodically or upon request.

The processor may determine or predict at least one executable operation of the terminal based on information determined or generated using data analysis and machine learning algorithms. To this end, the processor may control the terminal to execute a predicted operation or an operation determined to be desirable among the at least one executable operation.

Processors can perform a variety of functions that implement intelligent emulation (i.e., knowledge-based systems, inference systems, and knowledge acquisition systems). This can be applied to various types of systems (e.g., fuzzy logic systems), including adaptive systems, machine learning systems, artificial neural networks, etc.

The processor also involves speech and natural language speech processing, such as an I/O processing module, an environmental conditions module, a speech to text (STT) processing module, a natural language processing module, a workflow processing module, and a service processing module. It may include submodules that enable calculations.

The processor may receive image information (or a corresponding signal), audio information (or a corresponding signal), data, or user input information from the input unit.

Figure 2 is a flowchart for explaining a method of maintaining an air conditioner according to an embodiment of the present disclosure.

The terminal 100 operating as an air conditioner maintenance device according to an embodiment of the present disclosure may capture an image through a front camera after operating an augmented reality-based air conditioner maintenance service (S210). Before capturing an image, the terminal 100 generates unique identification information (ID) of the air conditioner 300 through short-distance communication with the air conditioner 300 or based on a marker such as a QR code or RF tag installed on the air conditioner 300. can be obtained.

In one embodiment, if the terminal 100 does not have a learning model capable of image recognition of the part corresponding to the identified air conditioner 300, the part corresponding to the air conditioner 300 identified by the air conditioner monitoring server 200 You can request transmission of a learning model capable of image recognition.

After acquiring the image captured by the camera (S220), the terminal 100 recognizes parts in the image captured by the air conditioner 300 through the camera based on a learning model suitable for the recognized air conditioner 300, and monitors the air conditioner. Operation history data of the recognized part may be requested from the server 200 or the air conditioner 300 (S230).

In another embodiment, after acquiring the unique identification information of the air conditioner 300, the terminal 100 sends the unique identification information to the air conditioner monitoring server 200 before the air conditioner 300 recognizes the part in the captured image (S220). and can receive operation history data of parts included in the air conditioner 300 (S230).

The terminal 100 may receive operation history data of all parts of the air conditioner 300 or operation history data of some parts from the air conditioner monitoring server 200 or the air conditioner 300.

When the terminal 100 receives operation history data of all parts of the air conditioner 300 from the air conditioner monitoring server 200, in one embodiment, the terminal 100 identifies the part in question from the air conditioner monitoring server 200. Information can be transmitted.

The terminal 100 may analyze the image to recognize parts in an image of the air conditioner 300 and recognize objects included in the image. At this time, the terminal 100 can apply the machine learning-based learning model received from the air conditioner monitoring server 200 to the image captured of the air conditioner 300 to recognize parts in the image. The learning model may be a deep learning method.

The learning model may include a neural network for image processing, and when a video or still image is input, it is trained to process a frame image of the still image or video to search for the bounding box of the part in the image. It may be a learning model, and may be a CNN (Convolution Neural Network) or R-CNN (Region based CNN) learning model. A learning model trained to recognize parts may be a learning model trained using parts images extracted from a database of air conditioner parts as training data. Before inputting the image taken of the air conditioner 300 to the learning model, the terminal 100 selectively performs image pre-processing such as converting the color image into a black and white image, histogram equalization, and image binarization. can do.

The terminal 100 displays the operation history data of the part received from the air conditioner monitoring server 200 or the air conditioner 300 along with an image of the air conditioner 300 using an augmented reality method near the part recognized in the image. You can do it (S250).

Referring to FIG. 3 , the terminal 100 may display an image captured of the outdoor unit 320 and display operation history data 115 of the component near the component of the outdoor unit 320 recognized in the image.

In one embodiment, when displaying the operation history data of a part in which a problem occurs, the terminal 100 displays a separate marker 111 on the recognized part so that the user (maintainer) can recognize the part in which the problem occurred. You can.

Figure 3 shows the example of displaying the operation history data of one part in the terminal 100, but as described above, the operation history data of all parts or a plurality of parts included in the outdoor unit 320 can be displayed together with the recognized parts. It is natural that it can be done.

The operation history data is operation data of a component related to the operating purpose and operating state of the component, and may be different for each outdoor unit 320 or indoor unit 310. Additionally, operation history data may differ depending on the model of the air conditioner.

The operation history data that can be displayed in the terminal 100 or that can be received from the air conditioner 300 in the air conditioner monitoring server 200 includes the outdoor unit's target high pressure, target low pressure, target superheating degree, target subcooling degree, compressor target frequency, compressor current frequency, Current FAN frequency, outside air temperature, current high pressure, current low pressure, suction temperature, discharge temperature, liquid pipe temperature, heat exchanger temperature, upper heat exchanger temperature, lower heat exchanger temperature, subcooling inlet temperature, subcooling outlet temperature, EEV opening degree, condensation temperature , evaporation temperature, compression ratio, inverter input current, inverter voltage, and FAN current may include some or all of the following.

The operation history data that can be displayed on the terminal 100 or received from the air conditioner 300 in the air conditioner monitoring server 200 includes the indoor unit operation mode, fan speed, rotation status, indoor unit LOCK, set temperature, indoor temperature, and indoor unit operation ON. /OFF, may include some or all of indoor unit error code, indoor pipe inlet temperature, indoor pipe outlet temperature, indoor unit capacity, and indoor unit operation rate.

The terminal 100 may display a plurality of operation history data related to one recognized part. For example, referring to FIG. 3 , reference data (eg, compressor target frequency) and currently changing data (eg, compressor current frequency) may be displayed together on the display. Therefore, the user can easily determine whether there is a problem with the corresponding part.

The terminal 100 can display the received operation history data of the component along with an image of the air conditioner 300 using an augmented reality method (S250).

The terminal 100 can transmit a display screen that displays operation history data of components using an augmented reality method along with images captured of the air conditioner 300 to the air conditioner monitoring server 200 in real time.

Accordingly, an expert located at a remote air conditioner monitoring center can determine the cause of the failure of the remote air conditioner 300 through a screen transmitted from the terminal 100 and transmit a response to it to the terminal 100.

In one embodiment, the air conditioner monitoring server 200 displays maintenance guide information preset in response to the fault code of the air conditioner 300 or maintenance guide information written by an expert based on the on-site screen transmitted from the terminal 100 to the terminal ( 100). At this time, the maintenance guide information may include a plurality of maintenance operations to be performed by the user (maintainer), and may include unique identification information of the parts subject to each maintenance operation. This is explained in detail below.

4 to 6, interfaces through which the terminal 100 displays operation history data of components on the display using an augmented reality method will be described in detail.

Figure 4 is an example diagram illustrating a method by which the terminal 100 displays past operation history data of a component on the display. Referring to FIG. 4 , the terminal 100 may display an interface 112 for displaying operation history data of a component operated at a previous time. The operation history data of the previous time may be operation history data near the time when a problem occurred in a part.

In one embodiment, the terminal 100 displays changes in the operation history data 113 according to the current time of the part on the display, and the user requests display of the operation history data at the time a problem occurs with the part through the interface 112. At the bottom (S260), the terminal 100 may transmit a message requesting transmission of unique identification information of the corresponding part and operation history data from a past point in time to the air conditioner monitoring server 200. Thereafter, the terminal 100 receives from the air conditioner monitoring server 200 the operation history data 115 within a preset section from the past of the part stored in the air conditioner monitoring server 200 or the time a problem occurred with the part. , this can be displayed on the display.

The air conditioner monitoring server 200 may receive operation history data of parts of the air conditioner 300 from the air conditioner 300 periodically, non-periodically, or in real time through a network and store it.

In one embodiment, the air conditioner monitoring server 200 may analyze the operation history data of the part and record the point in time when an abnormality occurs. In another embodiment, when the air conditioner monitoring server 200 receives an error code from the air conditioner 300, it may record the corresponding point in time. Therefore, when the terminal 100 requests operation history data at the time of occurrence of a problem with a part, the operation history data within a preset section from the recorded time of occurrence of the problem can be transmitted to the terminal 100.

The terminal 100 converts the operation history data 115 at the time of the problem of the part received from the air conditioner monitoring server 200 into a time-series as shown in FIG. 4 (b) so that changes over time can be determined. ) can be displayed to play dynamically on the display. Therefore, the user can check changes over time in the operation history data of the component at the time the problem occurred, which has the effect of easily and accurately determining the cause of the problem.

In one embodiment, when displaying past operation history data of a part, the terminal 100 may include an interface 114 that can display the current operation history data again or an interface 116 that can quickly replay the past operation history data being displayed. can be displayed on the display (S260). Therefore, the user can quickly check long-term past operation history data.

Figure 5 is an example diagram explaining how the terminal 100 displays a maintenance guide for an air conditioner on the display.

The terminal 100 may receive maintenance guide information for repairing a malfunction of the air conditioner 300 from the air conditioner monitoring server 200 and display the received maintenance guide information 511 and 513 on the display. The terminal 100 may sequentially display a maintenance guide configured to distinguish between a plurality of maintenance operations by the user, as shown in FIG. 5 .

The air conditioner monitoring server 200 may include unique identification information of parts corresponding to each maintenance operation among the plurality of maintenance operations in the maintenance guide information and transmit it to the terminal 100.

The terminal 100 uses an augmented reality method to place markers 520 and 530 indicating parts corresponding to the maintenance operation indicated in the maintenance guide on parts that match the unique identification information corresponding to the maintenance operation among the parts recognized in the image. It can be displayed. Therefore, even in the case of an air conditioner with which the user is less skilled in maintaining the air conditioner, the user can easily identify the parts corresponding to the maintenance operation of the maintenance guide, and thus can easily and quickly perform the air conditioner maintenance operation.

In one embodiment, when the maintenance guide information consists of a plurality of operations, the terminal 100 sequentially displays maintenance guide information 511 and 513 for the plurality of operations on the display, and along with the display of the guide information, the user performs the next action. Interfaces 512 and 514 that can request progress through maintenance operations can be displayed together. The terminal 100 may change and display the markers 520 and 530 indicating the parts subject to the maintenance operation according to a change in the maintenance operation.

Figure 6 is an example diagram explaining a method by which the terminal 100 displays past operation history data of a component on the display. Referring to FIG. 6, the terminal 100 can display (117) the time when a problem occurs in a component along with the operation history data of the component.

The air conditioner monitoring server 200 records the time when a problem with a part determined through analysis of the part's operation history data occurs or the time when an error code is received from the air conditioner 300, and sends the part's operation history data to the terminal 100. You can transmit the recorded time of problem occurrence.

The terminal 100 may display (117) the time when a problem occurs in a part along with the past operation history data of the part from the air conditioner monitoring server 200. Therefore, the user can quickly check the operation history data at the time a problem occurs in the part, thereby easily determining the cause of the part's failure.

The present invention described above can be implemented as computer-readable code on a program-recorded medium. Computer-readable media includes all types of recording devices that store data that can be read by a computer system. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SDD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. There is. Additionally, the computer may include a processor of the terminal.

Meanwhile, the program may be specially designed and configured for the present disclosure, or may be known and usable by those skilled in the art of computer software. Examples of programs may include not only machine language code such as that created by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

In the specification (particularly in the claims) of the present disclosure, the use of the term “above” and similar referential terms may refer to both the singular and the plural. In addition, when a range is described in the present disclosure, the invention includes the application of individual values within the range (unless there is a statement to the contrary), and each individual value constituting the range is described in the detailed description of the invention. It's the same.

Unless there is an explicit order or description to the contrary regarding the steps constituting the method according to the present disclosure, the steps may be performed in any suitable order. The present disclosure is not necessarily limited by the order of description of the steps above. The use of any examples or illustrative terms (e.g., etc.) in the present disclosure is merely to describe the present disclosure in detail, and unless limited by the claims, the scope of the present disclosure is limited by the examples or illustrative terms. It doesn't work. In addition, those skilled in the art will recognize that various modifications, combinations and changes may be made according to design conditions and factors within the scope of the appended claims or their equivalents.

Therefore, the spirit of the present disclosure should not be limited to the above-described embodiments, and the scope of the patent claims described below as well as all scopes equivalent to or equivalently changed from the claims are within the scope of the spirit of the present disclosure. It will be said to belong to

100: terminal 200: air conditioner monitoring server
300: air conditioner 310: indoor unit
320: Outdoor unit

Claims (19)

As a method of displaying air conditioner information in a terminal,
Obtaining an image including an air conditioner from the camera of the terminal;
Receiving operational history (historical) data of parts of the air conditioner;
displaying the acquired image on a display of the terminal; and
Comprising the step of matching the operation history data to the image including the air conditioner based on augmented reality and displaying it on the display,
The step of receiving the operation history data is,
It further includes receiving operation history data of a failed part of the air conditioner,
The step of displaying the operation history data on the display includes:
Further comprising displaying the operation history data including the operation history at the time of failure on the display along with an indication indicating the time of failure of the failed component,
How to display air conditioner information on the terminal.
delete According to claim 1,
The step of displaying the operation history data on the display includes:
Further comprising displaying operation history data of the faulty component within a predetermined time from the time of failure of the faulty component on the display,
How to display air conditioner information on the terminal.
According to claim 3,
The step of displaying the operation history data on the display includes:
Further comprising the step of reproducing and displaying changes over time in the operation history data of the failed component on the display in a time-series format,
How to display air conditioner information on the terminal.
According to claim 4,
The step of reproducing and displaying the operation history data on the display includes:
Further comprising displaying an interface capable of controlling the reproduction speed of the operation history data on the display,
How to display air conditioner information on the terminal.
According to claim 1,
Receiving maintenance guide information based on the operation history data of the defective part from a server; and
Further comprising displaying the maintenance guide information on the display,
How to display air conditioner information on the terminal.
According to claim 6,
The maintenance guide information includes a plurality of operation guide information instructing maintenance operations,
The guide information includes unique identification information of the part corresponding to the operation guide information,
How to display air conditioner information on the terminal.
According to claim 7,
Further comprising displaying an indication indicating a part corresponding to the unique identification information on the display,
How to display air conditioner information on the terminal.
delete A computer-readable recording medium storing a computer program configured to, when executed by a computer, cause the computer to execute the method of claim 1.
camera;
display;
processor; and
A memory electrically connected to the processor and storing at least one code executed by the processor,
The memory is,
When executed through the processor, the processor recognizes parts of the air conditioner in the image obtained by controlling the camera, receives historical data on the operation of the parts, and transmits the operation history data to the air conditioner. stores a code that matches the image based on augmented reality and causes it to be displayed on the display together with the image,
The operation history data includes operation history data of failed parts of the air conditioner,
The memory is,
further storing code that, when executed through the processor, causes the processor to display on the display the operating history data including the operating history at the time of the failure along with an indication indicating the time of failure of the failed component,
Augmented reality display device.
delete According to claim 11,
The memory is,
further storing code that, when executed through the processor, causes the processor to display on the display operation history data of the failed component within a predetermined time from the point of failure of the failed component,
Augmented reality display device.
According to claim 13,
The memory is,
further storing code that, when executed through the processor, causes the processor to reproduce and display changes over time in the operation history data of the failed component on the display in a time-series format,
Augmented reality display device.
According to claim 14,
The memory is,
further storing code that, when executed through the processor, causes the processor to display on the display an interface capable of controlling a playback speed of the operation history data,
Augmented reality display device.
According to claim 11,
The memory is,
further storing code that, when executed through the processor, causes the processor to receive maintenance guide information based on the operation history data of the failed part from a server and display the maintenance guide information on the display,
Augmented reality display device.
According to claim 16,
The maintenance guide information includes a plurality of operation guide information instructing maintenance operations,
The guide information includes unique identification information of the part corresponding to the operation guide information,
Augmented reality display device.
According to claim 17,
The memory is,
further storing code that, when executed through the processor, causes the processor to display an indicia on the display indicating a component corresponding to the unique identification information.
Augmented reality display device.
delete

Images