KR102550112B1 - SMART HOME APPLIANCE FOLLOW-UP MANAGEMENT SERVICE SYSTEM AND METHOD USING IoT SENSOR - Google Patents

Abstract

A follow-up management service system and method using an IoT sensor are disclosed. Smart home appliances that self-diagnose faults/errors, transmit self-diagnosis results, and receive software corresponding to the self-diagnosis results; and a follow-up management service cloud server that receives a self-diagnosis result from the smart appliance, transmits self-diagnosis-based analysis information to the manufacturer terminal, and transmits software according to the self-diagnosis result to the smart appliance, is configured to self-install by updating the received software. According to the post-management service system and method using the above-mentioned IoT sensor, MR content for self-repair is received and executed using MR smart glasses, and the consumer is configured to perform it directly, so that self-repair can be accurately and conveniently performed. There are possible effects.

Description

Post management service system and method using IoT sensor {SMART HOME APPLIANCE FOLLOW-UP MANAGEMENT SERVICE SYSTEM AND METHOD USING IoT SENSOR}

The present invention relates to a post management service system and method, and more specifically, to a post management service system and method using an IoT sensor.

Electronic products, flower pots, and refrigerator items are all subjects that require follow-up management in homes and offices.

Electronic products are repaired by visiting an A/S center or visiting a post-management person on a business trip when a malfunction or error occurs. In addition, since potted plants require different environments for each plant, the conditions such as the number of waterings, the amount of watering, and the amount of sunlight must be met, and each pot must be managed by optimizing them. In addition, the expiration date of food or ingredients in the refrigerator must be managed because refrigerator items are often thrown away after their expiration date.

As such, there are many targets for follow-up management for products, consumables, and consumer goods at home or offices, but it is not easy to efficiently manage these products or consumer goods.

On the other hand, in the face-to-face era, it is time for a paradigm shift for a new type of follow-up management.

Registered Patent Publication 10-2123975 Publication of Patent Publication 10-2011-0029028

An object of the present invention is to provide a post management service system using an IoT sensor.

Another object of the present invention is to provide a post management service method using an IoT sensor.

A follow-up management service system using an IoT sensor according to the object of the present invention described above includes a smart home appliance that self-diagnoses a failure/error, transmits a self-diagnosis result, and receives software corresponding to the self-diagnosis result; It may be configured to include a post management service cloud server that receives a self-diagnosis result from the smart appliance, transmits self-diagnosis-based analysis information to the manufacturer terminal, and transmits software according to the self-diagnosis result to the smart appliance. .

At this time, the smart home appliance may be configured to self-install by updating the received software.

A follow-up management service method using an IoT sensor according to another object of the present invention described above includes the steps of a smart home appliance self-diagnosing a failure/error and transmitting a self-diagnosis result to a post-management service cloud server; receiving, by the post management service cloud server, a self-diagnosis result from the smart home appliance; The post management service cloud server transmitting self-diagnosis-based analysis information to the manufacturer terminal and transmitting software according to the self-diagnosis result to the smart home appliance; The smart home appliance may be configured to include receiving software from the post management service cloud server.

At this time, the smart home appliance may be configured to further include updating the received software and self-installing the software.

According to the post-management service system and method using the above-mentioned IoT sensor, MR content for self-repair is received and executed using MR smart glasses, and the consumer is configured to perform it directly, so that self-repair can be accurately and conveniently performed. There are possible effects.

At this time, the smart appliance is configured to generate and display a QR code according to the self-diagnosis result, and to receive the corresponding MR content from the post management service cloud server by recognizing the QR code in the MR smart glasses, thereby preventing failure or error of the smart appliance. It has the effect of automatically extracting and executing necessary MR contents.

In addition, if necessary, it is configured to receive remote video post-management by performing a WebRTC video call with the post-management manager terminal using MR smart glasses, so that the post-management manager directly checks the product status through video and gives direct instructions to consumers Thus, there is an effect that the consumer can repair.

On the other hand, the home panel is configured to recognize the QR code of the fluorescent material of the pot, check the temperature, humidity, solar radiation, etc. from the IoT home sensor and instruct the watering, repotting, and nutrient cycles according to the specific plant in the pot, There is an effect of post-management of potted plants in the office or office.

In addition, by recognizing the barcodes of refrigerator items stored in the refrigerator, attaching the refrigerator item IoT tag to the refrigerator item and storing it, and detecting the refrigerator item IoT tag in real time to determine whether it is stored, the expiration date of each refrigerator item can be checked at home. There is an effect that can be managed ex post through the panel.

1 is a block diagram of a post management service system using an IoT sensor according to an embodiment of the present invention.
2 is a flowchart of a post management service method using an IoT sensor according to an embodiment of the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in specific contents for practicing the invention. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. Like reference numerals have been used for like elements throughout the description of each figure.

Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, the terms "include" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the present application, they should not be interpreted in an ideal or excessively formal meaning. don't

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a post management service system using an IoT sensor according to an embodiment of the present invention.

Referring to FIG. 1, the post management service system using the IoT sensor according to an embodiment of the present invention includes a refrigerator item IoT tag 100, a smart home appliance 200, a post management terminal 300, and MR smart glasses 400. ), an IoT home sensor 500, a fluorescent material QR code 600, a home panel 700, a manufacturer terminal 800, and a post management service cloud server 900.

Hereinafter, a detailed configuration will be described.

The refrigerator item IoT tag 100 is a configuration for attaching to the refrigerator item 10 .

The refrigerator item IoT tag 100 may be attached to a refrigerator food storage container, packaged material, milk container, or the like.

The smart home appliance 200 may be configured to self-diagnose malfunctions/errors, transmit the self-diagnosis results to the post management service cloud server 900, and generate and display a self-diagnosis QR code according to the self-diagnosis results. there is. The type of smart home appliance 100 is not limited, and TVs, refrigerators, washing machines, dishwashers, air purifiers, clothes care machines, vacuum cleaners, massage chairs, and the like may all be targets.

In addition, the smart home appliance 200 may be configured to generate a refrigerator item list using the refrigerator item IoT tag 100 and transmit it to the post management service cloud server 900 .

The smart home appliance 200 includes a sensor for fault/error confirmation 201, a sensor for parts replacement confirmation 202, a sensor for consumable replacement confirmation 203, a self-diagnosis module 204, a cloud interface middleware module 205, and a self-diagnosis module 204. Diagnostic QR code generation module (206), smart home appliance display module (207), smart home appliance Bluetooth module (208), software automatic update module (209), barcode sensor (210), refrigerator item IoT tag detection module (211), refrigerator It may be configured to include a product registration/update module 212, a refrigerator product database 213, and a refrigerator product list real-time transmission module 214.

Hereinafter, a detailed configuration will be described.

The failure/error checking sensor 201 may be configured to detect failure/error of the smart home appliance 200 .

The component replacement confirmation sensor 202 may be configured to detect whether or not parts of the smart home appliance 200 need to be replaced.

The sensor 203 for confirming replacement of consumables may be configured to detect whether or not replacement of consumables of the smart home appliance 200 is necessary. Consumables may include batteries, filters, and the like.

The self-diagnosis module 204 detects the failure/error detected by the sensor 201 for checking failure/error, the necessity of parts replacement detected by the sensor 202 for checking parts replacement, and the sensor 203 for checking consumable replacement. It may be configured to self-diagnose whether replacement of consumables is required.

The cloud interface middleware module 205 may be configured to transmit the self-diagnosis result of the self-diagnosis module 204 to the post management service cloud server 900 .

Also, the cloud interface middleware module 205 may be configured to receive software corresponding to a self-diagnosis result from the post management service cloud server 900 . As a result of self-diagnosis, if there is a software error or the version is low, a new version of the software can be received.

The self-diagnosis QR code generation module 206 may be configured to generate a self-diagnosis QR code including the self-diagnosis result of the self-diagnosis module 204 .

The smart home appliance display module 207 may be configured to display the self-diagnosis QR code generated by the self-diagnosis QR code generation module 206 .

The smart home appliance Bluetooth module 208 detects the failure/error detected by the sensor 201 for checking failure/error, the need for parts replacement detected by the sensor 202 for checking parts replacement, and the sensor 203 for checking consumable replacement. It may be configured to transmit, to the home panel 700, information on whether or not replacement of consumables is necessary in real time.

The software automatic update module 209 may be configured to perform automatic update using software received from the cloud interface middleware module 205 .

The barcode sensor 210 may be configured to recognize the barcode of the refrigerator item 10 .

It is possible to check the item and expiration date of the corresponding refrigerator item 10 by recognizing the barcode.

When the refrigerator item IoT tag 100 is attached to the refrigerator item 10, the refrigerator item IoT tag detection module 211 may be configured to detect tag-specific information of the attached refrigerator item IoT tag 100.

The refrigerator item registration/update module 212 may be configured to register item information and expiry date information of the barcode recognized by the barcode sensor 210 and unique tag information detected by the refrigerator item IoT tag detection module 211, It may be configured to update according to whether the refrigerator item 10 is stored in a refrigerator.

The refrigerator article database 213 may be configured to store item information, expiration date information, and unique tag information of the refrigerator article 10 registered by the refrigerator article registration/update module 212 .

In addition, the refrigerator article database 213 may be configured such that the stored item information, expiration date information, and tag-specific information are updated according to whether the refrigerator article 10 is stored in the refrigerator.

The refrigerator article list real-time transmission module 214 is configured to transmit the refrigerator article list of the refrigerator article 10 stored in the refrigerator article database 213 to the post management service cloud server 900 in real time through the cloud interface middleware module 205. It can be.

The post management manager terminal 300 may be configured to perform remote image post management in response to a failure/error.

The MR smart glasses 400 can be configured to recognize the self-diagnosis QR code displayed on the smart home appliance 2000, request, receive, and execute MR content for self-post management to the post-management service cloud server 900.

Meanwhile, the MR smart glasses 400 may be configured to perform a video call with the post management manager terminal 300 for remote image post management.

The MR smart glasses 400 includes a self-diagnosis QR code recognition module 401, a remote post-image management application module 402, an MR content request/reception module 403, an MR content storage module 404, and an MR content execution module ( 405), camera module 406, smart glasses display module 407, control module 408, WebRTC video call module 409, diagnosis/repair video clip real-time receiving module 410, PIP/VR content real-time It may be configured to include a conversion module 411 .

Hereinafter, a detailed configuration will be described.

The self-diagnosis QR code recognition module 401 may be configured to recognize the self-diagnosis result by recognizing the self-diagnosis QR code displayed on the smart appliance display module 207 of the smart appliance 200 .

The remote image post-management application module 402 automatically generates a remote image post-management application in response to the self-diagnosis result identified by the self-diagnosis QR code recognition module 401, and transfers the generated remote image post-management application to the post-management service cloud may be configured to transmit to the server 900.

The MR content request/reception module 403 may be configured to request and receive MR content from the post management service cloud server 900 for self follow-up management according to the self-diagnosis result identified by the self-diagnosis QR code recognition module 401. there is.

The MR content storage module 404 may be configured to store the MR content received in the MR content request/reception module 403.

The MR content execution module 405 may be configured to execute the MR content stored in the MR content storage module 404 for the corresponding smart home appliance 200 .

The camera module 406 may be configured to generate an image of the smart home appliance 200 for the MR content execution of the MR content execution module 405 .

The smart glasses display module 407 may be configured to display MR content executed in the MR content execution module 405 and an image generated by the camera module 406 in real time.

The smart glasses display module 406 may be configured to display MR content executed in the MR content execution module 404 and an image generated by the camera module 405 in real time. In the smart glass display module 406, the smart home appliance 100 is shown by the camera image, VR guide content is displayed as a guide for self-repair by MR content, and the shape of the smart home appliance 100 is grasped to (100) and display it in an augmented reality (AR) manner.

For example, if the door of the dishwasher is damaged and the door is replaced, the MR content execution module 404 executes VR content on how to replace the door delivered by courier, captures the dishwasher and the door from the camera image, and Each may be configured to be superimposed with AR content and displayed. Consumers can receive direct self-repair guides through the MR smart glasses 400 without having to search for a separate manual or make inquiries, and can proceed without errors.

If it is a massage chair, replacing pads for each part of the massage chair can also be easily replaced using the MR smart glasses 400.

The control module 408 may be configured to control the MR content execution of the MR content execution module 405 and image generation of the camera module 406 in real time.

The WebRTC video call module 409 may be configured to perform a WebRTC video call with the terminal 300 in charge of post management for remote video post management.

Specifically, the WebRTC video call module 409 transmits the video generated by the camera module 406 to the post management manager terminal 300 in real time, and receives and outputs post management data from the post management manager terminal 300. can be configured to

Here, the post-management data is auxiliary data for a post-management guide, and may be text, document, video, or the like.

In the case of WebRTC, if only a web browser is loaded on the MR smart glasses 400, the MR smart glasses 400 can make video calls and exchange data with the post management terminal 300 without a separate plug-in. can do.

In the case of remote video follow-up management using WebRTC video call, the person in charge of post-management can directly check the smart home appliance 100 and check the status through the camera image of the MR smart glasses 400, and allow the user to operate This is very useful because you can direct it to make repairs or repairs.

The diagnosis/repair video clip real-time receiving module 410 may be configured to receive a diagnosis/repair video clip for remote image post-management from the post-management service cloud server 900 in real time.

The PIP/VR contents real-time conversion module 411 converts the diagnosis/repair video clips received in real time from the diagnosis/repair video clip real-time receiving module 410 to VR contents in the form of PIP (picture-in-picture) during a WebRTC video call. It can be configured to convert and insert in real time. Consumers can perform self-repair after learning how to self-repair by executing the diagnosis/repair video clip inserted into the PIP.

The IoT home sensor 500 may be configured to detect and transmit temperature, humidity, and solar radiation in real time.

The fluorescent material QR code 600 is attached to the flowerpot 20 and may be configured to be identified by irradiating UV (ultra-violet) by forming a pattern with a fluorescent material. The fluorescent material QR code 600 does not affect aesthetics because the fluorescent material pattern is not normally visible, but can be identified through UV irradiation.

The home panel 700 may be configured to receive plant growth information of the flower pot 20 by recognizing the fluorescent material QR code 600 and to receive temperature, humidity, and solar radiation from the IoT home sensor 500 . The home panel 700 may also be configured as a tablet terminal.

Further, the home panel 700 may be configured to output plant growth control information of the pot 20 based on the received plant growth information and the received temperature, humidity, and solar radiation. Growing conditions are different for each plant in the pot 20, but it is not easy to match the amount of solar radiation as well as the number of water cycles, the amount of water, repotting, and nutrients. The home panel 700 may support performing post-management for various plants in the pot 20 .

Further, the home panel 700 may be configured to receive and output a refrigerator item list from the post management service cloud server 900 . Items in the refrigerator, expiry date can be output, and alarms can be output.

The home panel 700 includes a consumer terminal Bluetooth module 701, a smart home appliance IoT notification output module 702, a refrigerator item list receiving module 703, a refrigerator item expiration date notification output module 704, a QR code UV recognition module ( 705), plant nurturing information request/receive module 706, plant nurturing information storage module 707, IoT sensor data receiving module 708, IoT sensor data storage module 709, plant nurturing control module 710, home It may be configured to include a panel display module 711 .

Hereinafter, a detailed configuration will be described.

The consumer terminal Bluetooth module 701 may be configured to receive, in real time, information on failures/errors, parts replacement necessity, and consumable replacement necessity from the smart appliance Bluetooth module 208 of the smart appliance 200 .

The smart home appliance IoT notification output module 702 may be configured to output a notification according to information received in real time from the smart home appliance Bluetooth module 208 in real time.

The refrigerator article list receiving module 703 may be configured to receive a refrigerator article list from the post management service cloud server 900 .

The refrigerator article expiration date notification output module 704 may be configured to output a refrigerator article expiration date notification by using the refrigerator article list received from the refrigerator article list receiving module 703 .

The QR code UV recognition module 705 may be configured to identify the plant in the pot 20 by recognizing the fluorescent material QR code 600 .

The plant growth information request/reception module 706 may be configured to request and receive plant growth information about the plant identified by the QR code UV recognition module 705 from the post management service cloud server 900 .

The plant growth information storage module 707 may be configured to store the plant growth information received from the plant growth information request/reception module 706 .

The IoT sensor data receiving module 708 may be configured to receive temperature, humidity and solar radiation from the IoT home sensor 500 .

The IoT sensor data storage module 709 may be configured to store temperature, humidity, and solar radiation received from the IoT sensor data reception module 708 .

The plant growth control module 710 is configured to generate real-time plant growth control information in real time by using the plant growth information stored in the plant growth information storage module 707 and the temperature, humidity, and solar radiation stored in the IoT sensor data storage module 709. It can be.

The home home panel display module 711 may be configured to output real-time plant growth control information generated by the plant growth control module 710 in real time.

The manufacturer terminal 800 may be configured to receive and display self-diagnosis-based analysis information based on the self-diagnosis result of the smart home appliance 200 .

The post management service cloud server 900 may be configured to receive a self-diagnosis result from the smart home appliance 200 and transmit self-diagnosis-based analysis information to the manufacturer terminal 800 .

In addition, the post management service cloud server 900 receives the refrigerator item list from the smart home appliance 9200 and delivers it to the home panel 700, transmits MR content to the MR smart glass 400, and It may be configured to transmit real-time plant breeding information.

The post management service cloud server 900 may function as a platform for mediating post management services of various manufacturers.

The post management service cloud server 900 includes a self-diagnosis result receiving module 901, a self-diagnosis result database 902, a software error checking module 903, a parts life cycle calculation module 904, and a parts defect rate calculation module 905. , Self-diagnosis-based analysis information providing module (906), software transmission module (907), MR content database for smart appliances (908), MR content search module (909), MR content transmission module (910), remote image follow-up management scheduling module 911, diagnosis/repair video clip database 912, diagnosis/repair video clip real-time extraction module 913, diagnosis/repair video clip real-time transmission module 914, refrigerator item list real-time receiving module 915, home It may be configured to include a panel confirmation module 916, a refrigerator article list transmission module 917, a plant growth information database 918, a plant growth information search module 919, and a plant growth information transmission module 920.

Hereinafter, a detailed configuration will be described.

The self-diagnosis result receiving module 901 may be configured to receive a self-diagnosis result from the cloud interface middleware module 205 of the smart home appliance 200 .

The self-diagnosis result database 902 may be configured to store the self-diagnosis result received by the self-diagnosis result receiving module 901 .

The software error checking module 903 may be configured to check the software error of the smart home appliance 200 based on the self-diagnosis result stored in the self-diagnosis result database 902 .

The component life cycle calculation module 904 may be configured to calculate the component life cycle of the smart home appliance 200 based on the self-diagnosis results stored in the self-diagnosis result database 902 .

The component defect rate calculation module 905 may be configured to calculate the component defect rate of the smart home appliance 200 based on the self-diagnosis result stored in the self-diagnosis result database 902 .

The self-diagnosis-based analysis information providing module 906 includes software errors identified in the software error checking module 903, parts life cycles calculated in the parts life cycle calculation module 904, and parts calculated in the parts defect rate calculation module 905. It may be configured to provide self-diagnosis-based analysis information including a defect rate to the corresponding manufacturer terminal 800 .

The software transmission module 907 retrieves software for correcting the software error identified by the software error checking module 903 from a software database (not shown) and transmits the software through the cloud interface middleware module 205 of the smart home appliance 200. can be configured to send to the auto-update module 209.

The MR content database 908 for smart home appliances may be configured to store MR content for self-repair of the smart home appliance 200 in advance.

The MR content search module 909 may be configured to search for MR content from the MR content database 908 for smart home appliances according to the request of the MR content request/reception module 403 of the MR smart glasses 400.

The MR content transmission module 910 may be configured to transmit the MR content retrieved by the MR content search module 909 to the MR content request/reception module 403 of the MR smart glasses 400.

The remote post-management scheduling module 911 receives a request for remote post-management from the remote post-management application module 402 of the MR smart glasses 400 and establishes a communication between the terminal 300 in charge of post-management and the MR smart glasses 400. It can be configured to schedule remote picture follow-up.

The diagnosis/repair image clip database 912 may be configured to pre-store diagnosis/repair image clips for self follow-up management.

The diagnosis/repair image clip real-time extraction module 913 may be configured to extract diagnosis/repair image clips stored in the diagnosis/repair image clip database 912 in real time according to a request of the terminal 300 in charge of post-management.

The diagnosis/repair video clip real-time transmission module 914 transfers the diagnosis/repair video clip extracted in real-time from the diagnosis/repair video clip real-time extraction module 913 to the post management terminal 300 and the MR smart glasses 400 in a video call. It can be configured to transmit in real time.

The refrigerator article list real-time reception module 915 may be configured to receive the refrigerator article list from the refrigerator article list transmission module 214 of the smart home appliance 200 in real time.

The home panel checking module 916 may be configured to check the call panel 700 associated with the refrigerator item list received in real time from the refrigerator item list real time receiving module 915 .

The refrigerator article list transmission module 917 may be configured to transmit the refrigerator article list to the home panel 700 identified by the home panel confirmation module 916 .

The plant growth information database 918 may be configured to store plant growth information of various plants in advance.

The plant growth information search module 919 may be configured to search the plant growth information database 918 for plant growth information according to a request of the plant growth information request/reception module 706 of the home panel 700 .

The plant growth information transmission module 920 may be configured to transmit the plant growth information retrieved by the plant growth information search module 919 to the plant growth information request/reception module 706 of the home panel 700 .

2 is a flowchart of a post management service method using an IoT sensor according to an embodiment of the present invention.

Referring to FIG. 2 , the smart home appliance 200 self-diagnoses a failure/error and transmits a self-diagnosis result to the post management service cloud server 900 (S101).

Next, the post management service cloud server 900 receives a self-diagnosis result from the smart home appliance 200 (S104).

Next, the post management service cloud server 900 transmits self-diagnosis-based analysis information to the manufacturer terminal 800 and transmits software according to the self-diagnosis result to the smart home appliance 200 (S103).

Next, the smart home appliance 200 receives software from the post management service cloud server 900 (S104).

Next, the smart home appliance 200 self-installs by updating the received software (S105).

Although described with reference to the above embodiments, those skilled in the art can understand that the present invention can be variously modified and changed without departing from the spirit and scope of the present invention described in the claims below. There will be.

100: refrigerator item IoT tag
200: smart home appliances
201: sensor for fault/error confirmation
202: sensor for checking part replacement
203: sensor for confirming consumable replacement
204: self-diagnosis module
205: Cloud interface middleware module
206: self-diagnosis QR code generation module
207: smart home appliance display module
208: smart home appliance Bluetooth module
209: software automatic update module
210: barcode sensor
211 Refrigerator item IoT tag detection module
212: Refrigerator item registration/update module
213 Refrigerator item database
214 Refrigerator item list real-time transmission module
300: follow-up management terminal
400: MR smart glass
401: Self-diagnosis QR code recognition module
402: remote video follow-up management application module
403: MR content request/receive module
404: MR content storage module
405: MR content execution module
406: camera module
407: smart glass display module
408: control module
409: WebRTC video call module
410: diagnosis/repair image clip real-time receiving module
411: PIP/VR content real-time conversion module
500: IoT home sensor
600: fluorescent QR code
700: home panel
701: Consumer terminal Bluetooth module
702: smart home appliance IoT notification output module
703 Refrigerator item list receiving module
704 Refrigerator product expiry notification output module
705: QR code UV recognition module
706: Plant cultivation information request/reception module
707 Plant breeding information storage module
708: IoT sensor data receiving module
709: IoT sensor data storage module
710: plant breeding control module
711 Home panel display module
800: manufacturer terminal
900: post management service cloud server
901: Self-diagnosis result receiving module
902: Self-diagnosis result database
903: software error checking module
904: parts life cycle calculation module
905: component defect rate calculation module
906: Self-diagnosis based analysis information providing module
907: software transmission module
908: MR content database for smart home appliances
909: MR content search module
910: MR content transmission module
911: remote video follow-up management scheduling module
912 Diagnosis/repair image clip database
913: diagnosis/repair video clip real-time extraction module
914: Diagnosis/repair video clip real-time transmission module
915: Refrigerator item list real-time receiving module
916 Home panel confirmation module
917: refrigerator article list transmission module
918: plant cultivation information database
919: Plant cultivation information search module
920: Plant breeding information transmission module

Claims (4)

A refrigerator item IoT tag for attaching to a refrigerator item;
A smart home appliance that self-diagnoses malfunctions/errors and transmits self-diagnosis results, generates and displays a self-diagnosis QR code according to the self-diagnosis results, and generates and transmits a list of refrigerator items using the refrigerator item IoT tag;
a terminal in charge of post management that performs remote image post management in response to the failure/error;
MR smart glasses for recognizing the self-diagnosis QR code displayed on the smart home appliance, requesting, receiving, and executing MR content for self-post management, and performing a video call with the terminal in charge of post-management for the remote video post-management;
IoT home sensor that detects and transmits temperature, humidity and solar radiation in real time;
A fluorescent material QR code attached to a potted plant, patterned with a fluorescent material and identified by irradiating UV;
Recognizes the fluorescent material QR code to receive plant growth information of the pot, receive temperature, humidity, and solar radiation from the IoT home sensor, and based on the received plant growth information and the received temperature, humidity, and solar radiation a home panel that outputs plant growth control information of the potted plant and receives and outputs the list of refrigerator items;
a manufacturer terminal for receiving and displaying self-diagnosis-based analysis information based on a result of self-diagnosis of the smart home appliance;
Receiving a self-diagnosis result from the smart appliance, transmitting self-diagnosis-based analysis information to the manufacturer terminal, receiving a refrigerator item list from the smart appliance and delivering it to the home panel, and transmitting MR content to the MR smart glass And a follow-up management service cloud server for transmitting real-time plant growth information to the home panel,
The smart home appliance,
a failure/error checking sensor for detecting a failure/error of the smart home appliance;
a part replacement confirmation sensor for detecting whether or not parts of the smart home appliance need to be replaced;
a sensor for confirming replacement of consumables for detecting whether or not replacement of consumables in the smart home appliance is necessary;
A self-diagnosis module for self-diagnosing the failure/error detected by the failure/error confirmation sensor, the necessity of parts replacement detected by the component replacement confirmation sensor, and the need to replace consumables detected by the consumable replacement confirmation sensor;
a cloud interface middleware module for transmitting a self-diagnosis result of the self-diagnosis module to the post-management service cloud server and receiving software corresponding to the self-diagnosis result from the post-management service cloud server;
a self-diagnosis QR code generating module generating a self-diagnosis QR code including a self-diagnosis result of the self-diagnosis module;
A smart home appliance display module displaying the self-diagnosis QR code generated by the self-diagnosis QR code generation module;
Information on the failure/error detected by the failure/error confirmation sensor, the necessity of parts replacement detected by the component replacement confirmation sensor, and the need to replace consumables detected by the consumable replacement confirmation sensor are transmitted to the home panel in real time. A smart home appliance Bluetooth module that transmits;
an automatic software update module that receives software from the software transmission module of the post management service cloud server through the cloud interface middleware module and automatically updates the software using the received software;
a barcode sensor for recognizing the barcode of the refrigerator;
a refrigerator item IoT tag detection module for detecting tag-specific information of the refrigerator item IoT tag attached to the refrigerator item when the refrigerator item IoT tag is attached to the refrigerator item;
Refrigerator item registration/update module for registering item information and expiry date information of the barcode recognized by the barcode sensor and unique tag information detected by the refrigerator item IoT tag detection module, and updating according to whether or not the corresponding refrigerator item is stored in a refrigerator;
Item information, expiration date information, and tag-specific information of refrigerator articles registered by the refrigerator article registration/update module are stored, and the stored item information, expiration date information, and tag-specific information are updated according to whether the refrigerator articles are stored in a refrigerator. Refrigerator item database to be;
and a refrigerator article list real-time transmission module for transmitting a refrigerator article list of refrigerator articles stored in the refrigerator article database to the post management service cloud server in real time through the cloud interface middleware module,
The MR smart glass,
A self-diagnosis QR code recognition module that recognizes a self-diagnosis result by recognizing a self-diagnosis QR code displayed on a smart home appliance display module of the smart home appliance;
a remote image post-management application module that automatically generates a remote image post-management application corresponding to the self-diagnosis result identified by the self-diagnosis QR code recognition module and transmits it to the post-management service cloud server;
an MR content request/reception module for requesting and receiving MR content for self-post management from the post-management service cloud server according to the self-diagnosis result identified by the self-diagnosis QR code recognition module;
an MR contents storage module for storing the MR contents received by the MR contents request/reception module;
an MR content execution module that executes the MR content stored in the MR content storage module for a corresponding smart appliance;
a camera module generating an image of the smart home appliance for the MR content execution module to execute the MR content;
a smart glasses display module that displays the MR content executed by the MR content execution module and the image generated by the camera module in real time;
a control module that controls the MR content execution of the MR content execution module and image generation of the camera module in real time;
For remote image follow-up management, WebRTC video call is performed with the post-management manager terminal, the video generated by the camera module is transmitted in real time to the post-management manager terminal, and post-management data is received from the post-management manager terminal. WebRTC video call module to output by doing;
a diagnosis/repair video clip real-time receiving module for receiving a diagnosis/repair video clip for remote image post-management in real time from the post management service cloud server;
A PIP/VR contents real-time conversion module for converting and inserting the diagnosis/repair video clips received in real time from the diagnosis/repair video clip real-time reception module into PIP-type VR contents during the WebRTC video call,
The home panel,
a consumer terminal Bluetooth module that receives in real time information on failure/error, whether parts need to be replaced, and whether consumables need to be replaced, from the smart appliance Bluetooth module of the smart appliance;
a smart home appliance IoT notification output module for outputting a notification according to information received in real time from the smart home appliance Bluetooth module;
a refrigerator article list receiving module receiving a refrigerator article list from the post management service cloud server;
a refrigerator article expiration date notification module outputting a refrigerator article expiration date notification by using the refrigerator article list received by the refrigerator article list receiving module;
a QR code UV recognition module recognizing the fluorescent material QR code and recognizing the plant in the pot;
a plant growth information request/reception module for requesting and receiving plant growth information on the plant identified by the QR code UV recognition module from the post-management service cloud server;
a plant growth information storage module storing the plant growth information received from the plant growth information request/reception module;
IoT sensor data receiving module for receiving temperature, humidity and solar radiation from the IoT home sensor;
an IoT sensor data storage module for storing temperature, humidity, and solar radiation received from the IoT sensor data reception module;
A plant growth control module for generating real-time plant growth control information in real time using the plant growth information stored in the plant growth information storage module and the temperature, humidity, and solar radiation stored in the IoT sensor data storage module;
It is configured to include a home panel display module that outputs real-time plant growth control information generated in real time by the plant growth control module,
The post management service cloud server,
a self-diagnosis result receiving module for receiving a self-diagnosis result from the cloud interface middleware module of the smart home appliance;
a self-diagnosis result database storing the self-diagnosis result received by the self-diagnosis result receiving module;
a software error checking module for checking a software error of the smart appliance based on a self-diagnosis result stored in the self-diagnosis result database;
a component life cycle calculation module for calculating a component life cycle of the smart appliance based on a self-diagnosis result stored in the self-diagnosis result database;
a component defect rate calculation module for calculating a component defect rate of the smart appliance based on a self-diagnosis result stored in the self-diagnosis result database;
A person who provides self-diagnosis-based analysis information including the software error identified by the software error checking module, the component life cycle calculated by the component life cycle calculation module, and the component defect rate calculated by the component defect rate calculation module to the corresponding manufacturer's terminal Diagnostic-based analysis information providing module;
a software transmission module which retrieves software for correcting software errors identified by the software error checking module from a software database and transmits the software to the automatic software update module through a cloud interface middleware module of the smart appliance;
an MR content database for smart home appliances in which MR content for self-repair of the smart home appliance is previously stored;
an MR content search module for searching the MR content database for the smart home appliance for MR content according to a request of the MR content request/reception module of the MR smart glasses;
an MR content transmission module for transmitting the MR content retrieved by the MR content search module to the MR content request/reception module of the MR smart glasses;
a remote image post management scheduling module that receives a remote image post management request from the remote image post management application module of the MR smart glasses and schedules remote image post management between the post management manager terminal and the MR smart glasses;
A diagnosis/repair video clip database in which diagnosis/repair video clips are pre-stored for self-post management;
a diagnosis/repair video clip real-time extraction module for extracting a diagnosis/repair video clip stored in the diagnosis/repair video clip database in real time according to a request of the terminal in charge of post-management;
a diagnosis/repair video clip real-time transmission module that transmits the diagnosis/repair video clip extracted in real time from the diagnosis/repair video clip real-time extraction module to the MR smart glasses in a video call with the post management terminal;
a refrigerator article list real-time reception module for receiving a refrigerator article list from the refrigerator article list transmission module of the smart appliance in real time;
a home panel checking module for checking a home panel associated with the refrigerator article list received in real time from the refrigerator article list real time receiving module;
a refrigerator article list transmission module for transmitting the refrigerator article list to the home panel identified by the home panel confirmation module;
A plant development information database in which plant development information of various plants is stored in advance;
a plant growth information search module for retrieving plant growth information from the plant growth information database according to a request of the plant growth information request/reception module of the home panel;
A follow-up management service system using an IoT sensor, characterized in that it is configured to include a plant growth information transmission module for transmitting the plant growth information retrieved by the plant growth information search module to the plant growth information request/reception module of the home panel.
delete delete delete

Images