WO2017031033A1 - Augmented reality-based wiring, commissioning and monitoring of controllers - Google Patents

Augmented reality-based wiring, commissioning and monitoring of controllers Download PDF

Info

Publication number
WO2017031033A1
WO2017031033A1 PCT/US2016/046978 US2016046978W WO2017031033A1 WO 2017031033 A1 WO2017031033 A1 WO 2017031033A1 US 2016046978 W US2016046978 W US 2016046978W WO 2017031033 A1 WO2017031033 A1 WO 2017031033A1
Authority
WO
WIPO (PCT)
Prior art keywords
controller
data
building management
portable device
real
Prior art date
Application number
PCT/US2016/046978
Other languages
French (fr)
Inventor
Venugopala Kilingar Nadumane
Poornima Gubbi VENKATARAMU
Erwan Rivet
Graham Whiting
Original Assignee
Honeywell International Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Honeywell International Inc. filed Critical Honeywell International Inc.
Priority to EP16837633.3A priority Critical patent/EP3338240A4/en
Publication of WO2017031033A1 publication Critical patent/WO2017031033A1/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F24—HEATING; RANGES; VENTILATING
    • F24F—AIR-CONDITIONING; AIR-HUMIDIFICATION; VENTILATION; USE OF AIR CURRENTS FOR SCREENING
    • F24F11/00—Control or safety arrangements
    • F24F11/50—Control or safety arrangements characterised by user interfaces or communication
    • F24F11/56—Remote control
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/20—Administration of product repair or maintenance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/16—Real estate
    • G06Q50/163—Property management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F24—HEATING; RANGES; VENTILATING
    • F24F—AIR-CONDITIONING; AIR-HUMIDIFICATION; VENTILATION; USE OF AIR CURRENTS FOR SCREENING
    • F24F11/00—Control or safety arrangements
    • F24F11/30—Control or safety arrangements for purposes related to the operation of the system, e.g. for safety or monitoring
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F24—HEATING; RANGES; VENTILATING
    • F24F—AIR-CONDITIONING; AIR-HUMIDIFICATION; VENTILATION; USE OF AIR CURRENTS FOR SCREENING
    • F24F11/00—Control or safety arrangements
    • F24F11/50—Control or safety arrangements characterised by user interfaces or communication
    • F24F11/52—Indication arrangements, e.g. displays
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F24—HEATING; RANGES; VENTILATING
    • F24F—AIR-CONDITIONING; AIR-HUMIDIFICATION; VENTILATION; USE OF AIR CURRENTS FOR SCREENING
    • F24F11/00—Control or safety arrangements
    • F24F11/62—Control or safety arrangements characterised by the type of control or by internal processing, e.g. using fuzzy logic, adaptive control or estimation of values
    • G—PHYSICS
    • G08—SIGNALLING
    • G08C—TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
    • G08C2201/00—Transmission systems of control signals via wireless link
    • G08C2201/40—Remote control systems using repeaters, converters, gateways
    • G—PHYSICS
    • G08—SIGNALLING
    • G08C—TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
    • G08C2201/00—Transmission systems of control signals via wireless link
    • G08C2201/90—Additional features
    • G08C2201/93—Remote control using other portable devices, e.g. mobile phone, PDA, laptop
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/2803—Home automation networks
    • H04L12/2807—Exchanging configuration information on appliance services in a home automation network
    • H04L12/2809—Exchanging configuration information on appliance services in a home automation network indicating that an appliance service is present in a home automation network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/2803—Home automation networks
    • H04L12/2823—Reporting information sensed by appliance or service execution status of appliance services in a home automation network
    • H04L12/2825—Reporting to a device located outside the home and the home network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/131—Protocols for games, networked simulations or virtual reality

Definitions

  • the present disclosure pertains to controllers, and particularly to particular activities relative to controllers. Summary
  • the disclosure reveals a system for augmentation of reality-based components and actions related to controllers.
  • Actions may incorporate augmented reality-based wiring, commissioning and monitoring.
  • An augmented reality based application may run in a smart glass, head mounted display (HMD) or in a smart phone, which can augment, for example, a building management system controller and help in wiring, monitoring and commissioning of the controller.
  • HMD head mounted display
  • smart phone which can augment, for example, a building management system controller and help in wiring, monitoring and commissioning of the controller.
  • Figure 1 is a diagram of a system for augmentation of reality-based components and actions related to controllers
  • Figure 2 is a diagram of a system like that of Figure 1 but a client may be connected to a building controller and engineering tool;
  • Figure 3 is a diagram of a controller connected to a speech service module
  • Figure 4 is a diagram of a layout that may facilitate smart wiring
  • Figure 5 is a diagram table of example connection data. Description
  • the present system and approach may incorporate one or more processors, computers, controllers, user interfaces, wireless and/or wire connections, and/or the like, in an implementation described and/or shown herein.
  • This description may provide one or more illustrative and specific examples or ways of implementing the present system and approach. There may be numerous other examples or ways of implementing the system and approach.
  • each job site may consist of a minimum of one plant controller and unitary controllers to control different applications in a heating, ventilation and air conditioning (HVAC) system.
  • HVAC heating, ventilation and air conditioning
  • the controllers may be connected to different field devices through input/output (I/O) points of the controller.
  • I/O input/output
  • controllers may have a few onboard input/output (I/O) terminals and I/O modules.
  • Commissioning may be necessary to ensure that facilities, equipment, and systems are installed and function properly, and are successfully turned over to an owner. During commissioning, based on the engineering, one needs to make sure that all the I/O point terminals are connected to correct field devices. There may be a need to turn on and do fine tuning. For example, the air handler may be confirmed to be leveled. When expanding the facility or adding new field devices, it may be difficult to identify which terminals are connected to what. When a new pump is installed, the proper rotation and flow rate may be verified by changing the data point values.
  • the whole commissioning activity may be done manually by first doing all wiring and then check if all are connected properly by changing the data point and observing if it works. And there is not necessarily an easy way to visualize exactly which I/O point is connected to what and what are the values of it.
  • the present system and approach may be to have an augmented reality based application which will be running in a smart glass, head mounted display (HMD) or in a smart phone, which can augment the BMS controller.
  • HMD head mounted display
  • the client e.g., an app running in smart glass
  • the client may be operated to capture a real-time image of the BMS controller, and this image and other information from the client (such as a bar code, QR code (quick response code) or position information) may be used by the client to determine the identification of the exact controller.
  • the client may use a unique identification to request data such as controller information like device details, data points, and so forth from the building supervisory controller or commissioning tool.
  • an overlay of building management data may be displayed concurrently with a real-time image of the controller (e.g., a graphical and/or textual element positioned in the display near the equipment I/O terminals providing useful building management data).
  • a real-time image of the controller e.g., a graphical and/or textual element positioned in the display near the equipment I/O terminals providing useful building management data.
  • An operator of the client may then interact with the overlay objects (selectable icons/buttons or a graphical user interface (GUI)) to retrieve additional building management data such as use case-driven information that may incorporate providing the operator with an actuator-in-hand to tune a controller of the equipment.
  • overlay objects selective icons/buttons or a graphical user interface (GUI)
  • GUI graphical user interface
  • the operator may change the data point values by clicking on the values in the GUI.
  • the client may accept a voice input to change a value of a data point and augment the display with the action taken.
  • an action may incorporate Go to Analog Input 1, Write Analog Input 1 with 20, Read Analog Input 1, Read High Limit, Read Description, and so on.
  • a first case may be commissioning.
  • Client may use directly the controller or an engineering tool for fetching values and sending a command to the controller.
  • the client may augment the I/O terminals and display live values for each terminal.
  • the user may be able to select one data point and change the values, and check if everything works as intended.
  • the user may be able to use voice commands for changing the data point values. This may assist in commissioning activity and to better visualize the activity.
  • Figure 1 is a diagram of a system for augmentation of reality-based components and actions related to wiring, commissioning and monitoring of controllers.
  • a client 21 may be an application running in a monitor 28 such as smart glass, head mounted display, smart phone, or the like. Client 21 may have a connection with a web service 22. Web service 22 may be connected to a building supervisor 23. Building supervisor 23 may be connected to a building controller 24 and to a database 25.
  • a second case may be monitoring a building controller. If a building supervisor is already up and running, the building supervisor may act as a data server. If not, the client may access the data from the controller directly as in the first scenario. A client may fetch data from the data collected by the supervisor. It can also act as an interface to fetch real-time values from the controller and send commands to the controllers, such as changing set points, auto to manual override, and so on.
  • Figure 2 is a diagram of a system like that of Figure 1; however, client 21 may be connected to building controller 24 and to an engineering tool 26. Web service 22 may be situated in a building control 24 and in engineering tool 26. Further, database 25 may be situated in engineering tool 25. Client 21, as in Figure 1, may be an application running in a monitor 28 such as a smart glass, head mounted display, smart phone, or the like.
  • a user voice or audio captured by the smart glass/smart phone may be processed by the speech service which can internally use one of the existing speech API’s and generate the voice commands. Based on these commands, the client may connect to the controller and perform a required operation.
  • Figure 3 is a diagram of a controller 31 connected to a speech service module 32.
  • An audio capture module 33 may be connected to speech service module 32.
  • a microphone and speaker 34 may be connected to audio capture module 33. Voice messages and signals may be sent and received between the microphone and speaker 34 and controller 31 via audio capture module 33 and speech service module 32.
  • Augmented reality based application may be used in a portable device to augment the whole wiring process of a controller in a building management system (BMS).
  • BMS building management system
  • the wiring there may be a need to connect different IO terminals from the controller or IO modules to the field devices based on the engineering.
  • Commissioning may be necessary to ensure facilities, equipment, and systems are installed and function properly, and are successfully turned over to owner. During commissioning, based on the engineering, there may be a need to make sure that all the I/O point terminals are connected to correct field devices and turn on and do fine tuning. For example, the air handler may be confirmed to be leveled. When expanding the facility or adding new filed devices, it may be difficult to identify which terminals are connected to what. When a new pump is installed, the proper rotation and flow rate may be verified by changing the data point values.
  • the whole wiring and commissioning activity may be done manually by first doing all wiring by noting down which terminal is to be connected to which device and then being followed by a commissioning to check if all items are connected properly by changing the data point and observing if it works. And there is not necessarily an easy way to find out which terminals are to be connected to which field devices.
  • the present system and approach may have an augmented reality based application which can be running either in a smart glass, head mounted display (HMD) or in a smart phone, which will augment the whole process of wiring process.
  • HMD head mounted display
  • the client e.g., an app running in smart glass
  • the client may be operated to capture a real-time image of the BMS controller, and this image and other information from the client (such as a bar code, QR code (quick response code) or NFC (near field
  • tags may be used by the client to determine an identification of the exact controller.
  • the client may use the unique identification to request configuration data such as device details, field devices and terminal assignments, and so on from the controller. If these data are not available in the controller, then client may request data from the engineering tool.
  • the client may generate a workflow with a sequence of operation to be done during wiring activity based on the configuration and type of controller. It may augment the display concurrently a real-time image of the controller with a graphical and/or textual element positioned in the display near the equipment I/O terminals providing useful wiring information (for instance, a Connect Analog Output (AO)-1 to the Actuator Device #10). This may actually assist the installation person even though he/she does not necessarily have any information about the engineering. Also, if the installer is new to this field, it may assist the installer to learn the process.
  • AO Connect Analog Output
  • Figure 4 is a diagram of a layout that may facilitate smart wiring
  • a controller 41 may have various connections made to its terminals from field devices, actuators and sensors. Controller 41 may be connected to a client such as an application running in a smart glass, head mounted display, smart phone, and the like. Controller 41, after power and various connections to the controller, an indicator, such as a green LED, may turn on to indicate a level of satisfactory condition of the controller and essential connections. Items featured in Figure 4 may incorporate one or more controllers and field devices 42, configuration data 43 augmented reality 44 and controller live data 45. These items together may be provided for a smart wiring and commissioning advisor 46. Also, the items may also provide a basis for a monitoring advisor.
  • a user may need to enter details about the field devices which will be connected to the data points.
  • a table with all required connections may be created during a binary application generation by the engineering tool and be saved as a file in binary format where the application files are kept. And this file may be downloaded to the controller as part of a normal application download. .
  • Client may fetch table of connection data from the controller or tool and based on the table of connection data, a workflow may be created by the client and which can assist a wiring person in the process with step by step instructions that may incorporate the terminal, a type and description as illustrated with an example in a table 51 of Figure 5. Instances may involve: terminal 1, power, connect to 24VAC power; terminal 2, power, connect to 24VAC power; terminal 5, UI1 (universal input), connect to temperature sensor 1; terminal 6, AO1 (analog output), connect actuator device 10 to signal source 1; and so on.
  • For a corresponding sequence of steps may incorporate: 1) Connect terminals 1 and 2 to 24VAC power; 2) Connect terminals 3 and 4 to a MS/TP (Master-Slave/Token-Passing) channel; 3) Connect terminal 6 to actuator device #10; 4) ... ; and so forth.
  • MS/TP Master-Slave/Token-Passing
  • the sequences of operations and steps may be displayed in the screen and also played in a speaker or head phone as a voice output so that it can be easy for an installer to understand directions for wiring.
  • a client may accept a voice input for moving to next step. For instance,“Next to next Step”,“Skip this Step”,“Go to Previous Step”, and so on.
  • the steps may really assist in wiring activity and to better visualize the activity.
  • the user voice or audio may be captured by the smart glass / smart phone and be processed by a speech service which may internally use one of the existing speech API’s and generate the voice commands. Based on these commands, client may connect to the controller and perform the required operation.
  • the present system and approach may be applied to an Eagle TM controller in a Centraline TM platform.
  • An engineering tool used may be from Care TM .
  • a controller may be embellished to expose data.
  • the data may be used by an app running in a smart phone or smart glass to augment the wiring process.
  • the present system and approach may be applied to controller types other than the ones listed and a voice element may be incorporated in addition to others besides Eagle and Care.
  • the present system and approach may provide an intelligent wiring process, be combined with commissioning process, have better visualization, augment the BMS controller with real data, and provide natural ways to interact with another system such as an HVAC.
  • a system having an augmented reality based application for controllers may incorporate a device that captures a real-time image of a building management system controller, and information such as a bar or quick response code or position information to determine a unique identification (ID) of a controller.
  • the unique ID may be used to request data from a web service hosted as part of a building supervisory or commissioning tool, incorporating device details, and data points from a building management system database.
  • a real-time image of the building management system controller may be presented on a display. An interaction may be made with an overlay of objects on the real time image to retrieve additional data about the building management system controller. The additional data may be incorporated in an actuator-in-hand to tune the building management system controller.
  • the system may further incorporate a job site under supervision of the building management system controller that has at least one plant controller and one or more unitary controllers that control applications in a heating, ventilation and air conditioning (HVAC) system.
  • HVAC heating, ventilation and air conditioning
  • the system may further incorporate a mechanism that converts a voice input to a value change that augments the real-time image on the display.
  • An augmented reality based application may run in a mechanism selected from a group incorporating a smart glass, a head mounted display, a smart phone, a notebook, a tablet, a pad, and the like.
  • the building management system controller may be connected to one or more field devices through input and output (I/O) points of the building management system controller and the one or more field devices.
  • I/O input and output
  • a field device When a field device is installed and connected to the I/O data points of a controller, proper parameters of the field device may be verified by changing values at the I/O data points.
  • the real-time image of a controller may incorporate a graphical or a textual element positioned in the display adjacent to I/O terminals.
  • An overlay of building management data fetched with a web service may be displayed concurrently with the real-time image of the building management system controller.
  • the overlay of building management data may incorporate one or more items selected from a group incorporating objects, selectable icons, selectable buttons and graphical user interfaces.
  • the one or more items may enable a user to retrieve additional building management data.
  • Building management data may incorporate case-driven information that provides a user the actuator-in-hand to tune a controller of equipment.
  • Values of the data points may be changed by clicking on values shown on a graphical user interface, or values of the data points may be changed with a voice input to a mechanism for converting sound of voice inputs to actions that change values of the data points.
  • a mechanism having augmented reality-based wiring of controllers may incorporate a building management system having a controller, and a portable device.
  • the portable device may incorporate an application to augment a wiring process of the controller in the building management system.
  • a sequence of operation for the wiring process of the controller may be displayed through a web service on the portable device.
  • the sequence of operation may incorporate a download of application files from an engineering tool and the directions for the wiring process.
  • the wiring process may incorporate connecting input and output (I/O) terminals of the controller to field devices.
  • the application to augment the wiring process may be run in a device selected from a group incorporating a smart glass, a head mounted display, a smart phone, a tablet, a pad, a notebook, and the like.
  • the application may be run in the device to capture a real-time image of the controller.
  • the real-time image and other information may be to determine an identification of the controller.
  • the other information may be from one or more items selected from a group incorporating bar codes, quick response codes, near field communications, and web services.
  • the identification of the controller may be used to request configuration data of the controller through a web service.
  • the configuration data may be selected from a group incorporating device details, one or more field devices, and terminal assignments for the wiring process.
  • An approach of augmented reality based activity for a controller may incorporate augmenting a reality-based application, using the reality-based application in a portable device to augment a wiring process of a controller in a building management system, and displaying a workflow with a sequence of operation through a web service on the portable device.
  • the workflow with the sequence of operation may be based on a configuration or type of the controller.
  • the workflow with the sequence of operation may augment the wiring process of the controller.
  • the approach may further incorporate downloading application files to the portable device from an engineering tool to provide a basis for the wiring, wiring the controller with connections to one or more field devices through input/output terminals or input/output modules of the controller, and verifying each connection to the one or more field devices from the controller by changing a value of a data point at each input/output terminal the controller.
  • the portable device may be selected from a group incorporating a smart glass, a head mounted display, a smart phone, a tablet, a notebook, and the like.
  • the approach may further incorporate operating the reality-based application in the portable device to capture a real-time image of the controller, obtaining other information about one or more items selected from a group incorporating bar codes, quick response codes, and near field communication tags, using the real-time image and the other information with the portable device to determine a unique identification of the controller, and using the unique identification to request configuration data about the controller through a web service. If the configuration data are not available from the controller, then engineering data may be requested from the web service hosted as a part of an engineering tool. The engineering data may incorporate details of one or more field devices, the portable device, and assignments of the terminals of the controller.
  • the one or more field devices may support a heating, ventilation and air conditioning (HVAC) system.
  • HVAC heating, ventilation and air conditioning
  • the approach may further incorporate generating the workflow with the sequence of operation for augmenting the wiring process based on the configuration or type of the controller.
  • the workflow with the sequence of operation may augment the displaying of the portable device concurrently of the real-time image of the controller with a graphical and/or textual element positioned in the display near the input/output terminal providing wiring information.
  • the real-time image of the controller may augment the input/output terminals and show live data point values at the input/output terminals.
  • a live data point value may be selected and be changed.
  • a live data point value may be changed with voice commands via a speech processor to the portable device.
  • the file with the table may be downloaded to the controller as part of a normal download of an application.
  • the controller may expose data of the file through a web service over a net.
  • the controller cannot expose the data of the file, then the file may be exposed by the engineering tool through a sub service.
  • the work flow may assist a wiring process with step-by-step instructions.
  • the instructions may be displayed on a screen of the display of the portable device or played as a voice output on the portable device.

Abstract

A system for augmentation of reality-based components and actions related to controllers. Actions may incorporate augmented reality-based wiring, commissioning and monitoring. An augmented reality based application may run in a smart glass, head mounted display (HMD) or in a smart phone, which can augment, for example, a building management system controller and help in wiring, monitoring and commissioning of the controller.

Description

AUGMENTED REALITY-BASED WIRING, COMMISSIONING AND MONITORING OF CONTROLLERS Background
The present disclosure pertains to controllers, and particularly to particular activities relative to controllers. Summary
The disclosure reveals a system for augmentation of reality-based components and actions related to controllers. Actions may incorporate augmented reality-based wiring, commissioning and monitoring. An augmented reality based application may run in a smart glass, head mounted display (HMD) or in a smart phone, which can augment, for example, a building management system controller and help in wiring, monitoring and commissioning of the controller. Brief Description of the Drawing
Figure 1 is a diagram of a system for augmentation of reality-based components and actions related to controllers;
Figure 2 is a diagram of a system like that of Figure 1 but a client may be connected to a building controller and engineering tool;
Figure 3 is a diagram of a controller connected to a speech service module; Figure 4 is a diagram of a layout that may facilitate smart wiring,
commissioning and monitoring of controllers and other equipment; and
Figure 5 is a diagram table of example connection data. Description
The present system and approach may incorporate one or more processors, computers, controllers, user interfaces, wireless and/or wire connections, and/or the like, in an implementation described and/or shown herein.
This description may provide one or more illustrative and specific examples or ways of implementing the present system and approach. There may be numerous other examples or ways of implementing the system and approach.
In BMS projects, each job site may consist of a minimum of one plant controller and unitary controllers to control different applications in a heating, ventilation and air conditioning (HVAC) system. The controllers may be connected to different field devices through input/output (I/O) points of the controller. Often, controllers may have a few onboard input/output (I/O) terminals and I/O modules.
Commissioning may be necessary to ensure that facilities, equipment, and systems are installed and function properly, and are successfully turned over to an owner. During commissioning, based on the engineering, one needs to make sure that all the I/O point terminals are connected to correct field devices. There may be a need to turn on and do fine tuning. For example, the air handler may be confirmed to be leveled. When expanding the facility or adding new field devices, it may be difficult to identify which terminals are connected to what. When a new pump is installed, the proper rotation and flow rate may be verified by changing the data point values.
Outside of the present system and approach, the whole commissioning activity may be done manually by first doing all wiring and then check if all are connected properly by changing the data point and observing if it works. And there is not necessarily an easy way to visualize exactly which I/O point is connected to what and what are the values of it.
The present system and approach may be to have an augmented reality based application which will be running in a smart glass, head mounted display (HMD) or in a smart phone, which can augment the BMS controller.
During a commissioning operation of a building management system (BMS), the client (e.g., an app running in smart glass) may be operated to capture a real-time image of the BMS controller, and this image and other information from the client (such as a bar code, QR code (quick response code) or position information) may be used by the client to determine the identification of the exact controller. The client may use a unique identification to request data such as controller information like device details, data points, and so forth from the building supervisory controller or commissioning tool.
Once the client receives this information, an overlay of building management data, then it may be displayed concurrently with a real-time image of the controller (e.g., a graphical and/or textual element positioned in the display near the equipment I/O terminals providing useful building management data).
An operator of the client may then interact with the overlay objects (selectable icons/buttons or a graphical user interface (GUI)) to retrieve additional building management data such as use case-driven information that may incorporate providing the operator with an actuator-in-hand to tune a controller of the equipment.
The operator may change the data point values by clicking on the values in the GUI. Also, the client may accept a voice input to change a value of a data point and augment the display with the action taken. For example, an action may incorporate Go to Analog Input 1, Write Analog Input 1 with 20, Read Analog Input 1, Read High Limit, Read Description, and so on.
A first case may be commissioning. Client may use directly the controller or an engineering tool for fetching values and sending a command to the controller. During commissioning, when client captures a real-time image of the BMS controller, the client may augment the I/O terminals and display live values for each terminal. The user may be able to select one data point and change the values, and check if everything works as intended. Also, the user may be able to use voice commands for changing the data point values. This may assist in commissioning activity and to better visualize the activity.
Figure 1 is a diagram of a system for augmentation of reality-based components and actions related to wiring, commissioning and monitoring of controllers. A client 21 may be an application running in a monitor 28 such as smart glass, head mounted display, smart phone, or the like. Client 21 may have a connection with a web service 22. Web service 22 may be connected to a building supervisor 23. Building supervisor 23 may be connected to a building controller 24 and to a database 25.
A second case may be monitoring a building controller. If a building supervisor is already up and running, the building supervisor may act as a data server. If not, the client may access the data from the controller directly as in the first scenario. A client may fetch data from the data collected by the supervisor. It can also act as an interface to fetch real-time values from the controller and send commands to the controllers, such as changing set points, auto to manual override, and so on.
Figure 2 is a diagram of a system like that of Figure 1; however, client 21 may be connected to building controller 24 and to an engineering tool 26. Web service 22 may be situated in a building control 24 and in engineering tool 26. Further, database 25 may be situated in engineering tool 25. Client 21, as in Figure 1, may be an application running in a monitor 28 such as a smart glass, head mounted display, smart phone, or the like.
A user voice or audio captured by the smart glass/smart phone may be processed by the speech service which can internally use one of the existing speech API’s and generate the voice commands. Based on these commands, the client may connect to the controller and perform a required operation.
Figure 3 is a diagram of a controller 31 connected to a speech service module 32. An audio capture module 33 may be connected to speech service module 32. A microphone and speaker 34 may be connected to audio capture module 33. Voice messages and signals may be sent and received between the microphone and speaker 34 and controller 31 via audio capture module 33 and speech service module 32.
Augmented reality based application may be used in a portable device to augment the whole wiring process of a controller in a building management system (BMS). During engineering, there may be a need to download the binary application files from the engineering tool followed by wiring. During the wiring, there may be a need to connect different IO terminals from the controller or IO modules to the field devices based on the engineering.
Once a physical wiring process is completed, there may be a need to perform a wiring check, where an installer has to verify that everything is connected properly. For this, an installer may need to check the connectivity, remove a connection and check if correct terminal is connected.
Commissioning may be necessary to ensure facilities, equipment, and systems are installed and function properly, and are successfully turned over to owner. During commissioning, based on the engineering, there may be a need to make sure that all the I/O point terminals are connected to correct field devices and turn on and do fine tuning. For example, the air handler may be confirmed to be leveled. When expanding the facility or adding new filed devices, it may be difficult to identify which terminals are connected to what. When a new pump is installed, the proper rotation and flow rate may be verified by changing the data point values.
In absence of the present system and approach, the whole wiring and commissioning activity may be done manually by first doing all wiring by noting down which terminal is to be connected to which device and then being followed by a commissioning to check if all items are connected properly by changing the data point and observing if it works. And there is not necessarily an easy way to find out which terminals are to be connected to which field devices.
The present system and approach may have an augmented reality based application which can be running either in a smart glass, head mounted display (HMD) or in a smart phone, which will augment the whole process of wiring process.
During controller installation/wiring and commissioning of a building management system of a controller and field devices in building management system, the client (e.g., an app running in smart glass) may be operated to capture a real-time image of the BMS controller, and this image and other information from the client (such as a bar code, QR code (quick response code) or NFC (near field
communication) tags) may be used by the client to determine an identification of the exact controller. The client may use the unique identification to request configuration data such as device details, field devices and terminal assignments, and so on from the controller. If these data are not available in the controller, then client may request data from the engineering tool.
Once the client receives this information, it may generate a workflow with a sequence of operation to be done during wiring activity based on the configuration and type of controller. It may augment the display concurrently a real-time image of the controller with a graphical and/or textual element positioned in the display near the equipment I/O terminals providing useful wiring information (for instance, a Connect Analog Output (AO)-1 to the Actuator Device #10). This may actually assist the installation person even though he/she does not necessarily have any information about the engineering. Also, if the installer is new to this field, it may assist the installer to learn the process.
Figure 4 is a diagram of a layout that may facilitate smart wiring,
commissioning and monitoring of controllers and other equipment. A controller 41 may have various connections made to its terminals from field devices, actuators and sensors. Controller 41 may be connected to a client such as an application running in a smart glass, head mounted display, smart phone, and the like. Controller 41, after power and various connections to the controller, an indicator, such as a green LED, may turn on to indicate a level of satisfactory condition of the controller and essential connections. Items featured in Figure 4 may incorporate one or more controllers and field devices 42, configuration data 43 augmented reality 44 and controller live data 45. These items together may be provided for a smart wiring and commissioning advisor 46. Also, the items may also provide a basis for a monitoring advisor.
During data point creation and terminal assignment with the engineering tool; mandatorily, a user may need to enter details about the field devices which will be connected to the data points. Based on the configuration, a table with all required connections may be created during a binary application generation by the engineering tool and be saved as a file in binary format where the application files are kept. And this file may be downloaded to the controller as part of a normal application download. .
Client may fetch table of connection data from the controller or tool and based on the table of connection data, a workflow may be created by the client and which can assist a wiring person in the process with step by step instructions that may incorporate the terminal, a type and description as illustrated with an example in a table 51 of Figure 5. Instances may involve: terminal 1, power, connect to 24VAC power; terminal 2, power, connect to 24VAC power; terminal 5, UI1 (universal input), connect to temperature sensor 1; terminal 6, AO1 (analog output), connect actuator device 10 to signal source 1; and so on. For a corresponding sequence of steps may incorporate: 1) Connect terminals 1 and 2 to 24VAC power; 2) Connect terminals 3 and 4 to a MS/TP (Master-Slave/Token-Passing) channel; 3) Connect terminal 6 to actuator device #10; 4) ... ; and so forth.
The sequences of operations and steps may be displayed in the screen and also played in a speaker or head phone as a voice output so that it can be easy for an installer to understand directions for wiring. During each step, a client may accept a voice input for moving to next step. For instance,“Next to next Step”,“Skip this Step”,“Go to Previous Step”, and so on. The steps may really assist in wiring activity and to better visualize the activity.
The user voice or audio may be captured by the smart glass / smart phone and be processed by a speech service which may internally use one of the existing speech API’s and generate the voice commands. Based on these commands, client may connect to the controller and perform the required operation.
The present system and approach may be applied to an EagleTM controller in a CentralineTM platform. An engineering tool used may be from CareTM. And a controller may be embellished to expose data. The data may be used by an app running in a smart phone or smart glass to augment the wiring process. The present system and approach may be applied to controller types other than the ones listed and a voice element may be incorporated in addition to others besides Eagle and Care.
The present system and approach may provide an intelligent wiring process, be combined with commissioning process, have better visualization, augment the BMS controller with real data, and provide natural ways to interact with another system such as an HVAC.
To recap, a system having an augmented reality based application for controllers, may incorporate a device that captures a real-time image of a building management system controller, and information such as a bar or quick response code or position information to determine a unique identification (ID) of a controller. The unique ID may be used to request data from a web service hosted as part of a building supervisory or commissioning tool, incorporating device details, and data points from a building management system database. Upon receipt of data, a real-time image of the building management system controller may be presented on a display. An interaction may be made with an overlay of objects on the real time image to retrieve additional data about the building management system controller. The additional data may be incorporated in an actuator-in-hand to tune the building management system controller.
The system may further incorporate a job site under supervision of the building management system controller that has at least one plant controller and one or more unitary controllers that control applications in a heating, ventilation and air conditioning (HVAC) system.
The system may further incorporate a mechanism that converts a voice input to a value change that augments the real-time image on the display. An augmented reality based application may run in a mechanism selected from a group incorporating a smart glass, a head mounted display, a smart phone, a notebook, a tablet, a pad, and the like.
The building management system controller may be connected to one or more field devices through input and output (I/O) points of the building management system controller and the one or more field devices. When a field device is installed and connected to the I/O data points of a controller, proper parameters of the field device may be verified by changing values at the I/O data points.
The real-time image of a controller may incorporate a graphical or a textual element positioned in the display adjacent to I/O terminals. An overlay of building management data fetched with a web service may be displayed concurrently with the real-time image of the building management system controller.
The overlay of building management data may incorporate one or more items selected from a group incorporating objects, selectable icons, selectable buttons and graphical user interfaces. The one or more items may enable a user to retrieve additional building management data. Building management data may incorporate case-driven information that provides a user the actuator-in-hand to tune a controller of equipment.
Values of the data points may be changed by clicking on values shown on a graphical user interface, or values of the data points may be changed with a voice input to a mechanism for converting sound of voice inputs to actions that change values of the data points.
A mechanism having augmented reality-based wiring of controllers, may incorporate a building management system having a controller, and a portable device. The portable device may incorporate an application to augment a wiring process of the controller in the building management system. A sequence of operation for the wiring process of the controller may be displayed through a web service on the portable device. The sequence of operation may incorporate a download of application files from an engineering tool and the directions for the wiring process. The wiring process may incorporate connecting input and output (I/O) terminals of the controller to field devices.
The application to augment the wiring process may be run in a device selected from a group incorporating a smart glass, a head mounted display, a smart phone, a tablet, a pad, a notebook, and the like.
The application may be run in the device to capture a real-time image of the controller. The real-time image and other information may be to determine an identification of the controller. The other information may be from one or more items selected from a group incorporating bar codes, quick response codes, near field communications, and web services.
The identification of the controller may be used to request configuration data of the controller through a web service. The configuration data may be selected from a group incorporating device details, one or more field devices, and terminal assignments for the wiring process. An approach of augmented reality based activity for a controller, may incorporate augmenting a reality-based application, using the reality-based application in a portable device to augment a wiring process of a controller in a building management system, and displaying a workflow with a sequence of operation through a web service on the portable device. The workflow with the sequence of operation may be based on a configuration or type of the controller. The workflow with the sequence of operation may augment the wiring process of the controller.
The approach may further incorporate downloading application files to the portable device from an engineering tool to provide a basis for the wiring, wiring the controller with connections to one or more field devices through input/output terminals or input/output modules of the controller, and verifying each connection to the one or more field devices from the controller by changing a value of a data point at each input/output terminal the controller.
The portable device may be selected from a group incorporating a smart glass, a head mounted display, a smart phone, a tablet, a notebook, and the like.
The approach may further incorporate operating the reality-based application in the portable device to capture a real-time image of the controller, obtaining other information about one or more items selected from a group incorporating bar codes, quick response codes, and near field communication tags, using the real-time image and the other information with the portable device to determine a unique identification of the controller, and using the unique identification to request configuration data about the controller through a web service. If the configuration data are not available from the controller, then engineering data may be requested from the web service hosted as a part of an engineering tool. The engineering data may incorporate details of one or more field devices, the portable device, and assignments of the terminals of the controller. The one or more field devices may support a heating, ventilation and air conditioning (HVAC) system.
The approach may further incorporate generating the workflow with the sequence of operation for augmenting the wiring process based on the configuration or type of the controller. The workflow with the sequence of operation may augment the displaying of the portable device concurrently of the real-time image of the controller with a graphical and/or textual element positioned in the display near the input/output terminal providing wiring information. The real-time image of the controller may augment the input/output terminals and show live data point values at the input/output terminals. A live data point value may be selected and be changed. A live data point value may be changed with voice commands via a speech processor to the portable device.
During data point creation and terminal assignment with the engineering tool, details about the field devices which are connected to the data points may be entered by a user. Based on the configuration, a table with virtually all required connections may be created during an application by the engineering tool and saved as a file in a binary format where the application files are stored.
The file with the table may be downloaded to the controller as part of a normal download of an application. When the controller has the file with the table, the controller may expose data of the file through a web service over a net. When the controller cannot expose the data of the file, then the file may be exposed by the engineering tool through a sub service.
Based on the table with virtually all required connections, the work flow may assist a wiring process with step-by-step instructions. The instructions may be displayed on a screen of the display of the portable device or played as a voice output on the portable device.
Any publication or patent document noted herein is hereby incorporated by reference to the same extent as if each individual publication or patent document was specifically and individually indicated to be incorporated by reference.
In the present specification, some of the matter may be of a hypothetical or prophetic nature although stated in another manner or tense.
Although the present system and/or approach has been described with respect to at least one illustrative example, many variations and modifications will become apparent to those skilled in the art upon reading the specification. It is therefore the intention that the appended claims be interpreted as broadly as possible in view of the related art to include all such variations and modifications.

Claims

What is claimed is:
1. A system having an augmented reality based application for controllers, comprising:
a device that captures a real-time image of a building management system controller, and information such as a bar or quick response code or position information to determine a unique identification (ID) of a controller; and
wherein:
the unique ID is used to request data from a web service hosted as part of a building supervisory or commissioning tool, comprising device details, and data points from a building management system database;
upon receipt of data, a real-time image of the building management system controller is presented on a display;
an interaction is made with an overlay of objects on the real time image to retrieve additional data about the building management system controller; and
the additional data are incorporated in an actuator-in-hand to tune the building management system controller.
2. The system of claim 1, further comprising a job site under supervision of the building management system controller that has at least one plant controller and one or more unitary controllers that control applications in a heating, ventilation and air conditioning (HVAC) system.
3. The system of claim 1, further comprising:
a mechanism that converts a voice input to a value change that
augments the real-time image on the display; and wherein an augmented reality based application runs in a mechanism selected from a group comprising a smart glass, a head mounted display, a tablet, a notebook, a pad, and a smart phone.
4. The system of claim 1, wherein:
the building management system controller is connected to one or more field devices through input and output (I/O) points of the building management system controller and the one or more field devices; and
when a field device is installed and connected to the I/O data points of a controller, proper parameters of the field device are verified by changing values at the I/O data points.
5. The system of claim 1, wherein:
the real-time image of a controller comprises a graphical or a textual element positioned in the display adjacent to I/O terminals; an overlay of building management data fetched with a web service is displayed concurrently with the real-time image of the building management system controller;
the overlay of building management data comprises one or more items selected from a group comprising objects, selectable icons, selectable buttons and graphical user interfaces; the one or more items enable a user to retrieve additional building management data; and
building management data comprise case-driven information that provides a user the actuator-in-hand to tune a controller of equipment.
6. The system of claim 4, wherein:
values of the data points are changed by clicking on values shown on a graphical user interface; or
values of the data points are changed with a voice input to a
mechanism for converting sound of voice inputs to actions that change values of the data points.
7. A mechanism having augmented reality-based wiring of controllers, comprising:
a building management system having a controller; and a portable device; and
wherein:
the portable device comprises an application to augment a wiring process of the controller in the building management system; a sequence of operation for the wiring process of the controller is displayed through a web service on the portable device;
the sequence of operation comprises a download of application files from an engineering tool and the directions for the wiring process; and
the wiring process comprises connecting input and output (I/O)
terminals of the controller to field devices.
8. The mechanism of claim 7, wherein the application to augment the wiring process can be run in a device selected from a group comprising a smart glass, a head mounted display, a tablet, a notebook, a pad, and a smart phone.
9. The mechanism of claim 7, wherein:
the application is run in the device to capture a real-time image of the controller;
the real-time image and other information are to determine an
identification of the controller;
the other information is from one or more items selected from a group comprising bar codes, quick response codes, near field communications, and web services;
the identification of the controller is used to request configuration data of the controller through a web service; and
the configuration data are selected from a group comprising device details, one or more field devices, and terminal assignments for the wiring process.
10. A method of augmented reality based activity for a controller, comprising:
augmenting a reality-based application; using the reality-based application in a portable device to augment a wiring process of a controller in a building management system; and
displaying a workflow with a sequence of operation through a web service on the portable device; and
wherein:
the workflow with the sequence of operation is based on a
configuration or type of the controller; and
the workflow with the sequence of operation augments the wiring process of the controller.
11. The method of claim 10, further comprising:
downloading application files to the portable device from an
engineering tool to provide a basis for the wiring; wiring the controller with connections to one or more field devices through input/output terminals or input/output modules of the controller; and
verifying each connection to the one or more field devices from the controller by changing a value of a data point at each input/output terminal the controller.
12. The method of claim 10, wherein the portable device is selected from a group comprising a smart glass, a head mounted display, a smart phone, a pad, a tablet, and a notebook.
13. The method of claim 10, further compromising:
operating the reality-based application in the portable device to capture a real-time image of the controller;
obtaining other information about one or more items selected from a group comprising bar codes, quick response codes, and near field communication tags;
using the real-time image and the other information with the portable device to determine a unique identification of the controller; and using the unique identification to request configuration data about the controller through a web service; and
wherein:
if the configuration data are not available from the controller, then engineering data are requested from the web service hosted as a part of an engineering tool;
the engineering data comprise details of one or more field devices, the portable device, and assignments of the terminals of the controller; and
the one or more field devices support a heating, ventilation and air conditioning (HVAC) system.
14. The method of claim 10, further comprising:
generating the workflow with the sequence of operation for
augmenting the wiring process based on the configuration or type of the controller; and
wherein:
the workflow with the sequence of operation augments the displaying of the portable device concurrently of the real-time image of the controller with a graphical and/or textual element positioned in the display near the input/output terminal providing wiring information;
the real-time image of the controller augments the input/output
terminals and shows live data point values at the input/output terminals;
a live data point value can be selected and be changed; and a live data point value can be changed with voice commands via a speech processor to the portable device.
15. The method of claim 13, wherein:
during data point creation and terminal assignment with the
engineering tool, details about the field devices which are connected to the data points are entered by a user; based on the configuration, a table with virtually all required connections is created during an application by the engineering tool and saved as a file in a binary format where the application files are stored;
based on the table with virtually all required connections, the work flow can assist a wiring process with step-by-step instructions; and
the instructions can be displayed on a screen of the display of the portable device or played as a voice output on the portable device.
PCT/US2016/046978 2015-08-19 2016-08-15 Augmented reality-based wiring, commissioning and monitoring of controllers WO2017031033A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP16837633.3A EP3338240A4 (en) 2015-08-19 2016-08-15 Augmented reality-based wiring, commissioning and monitoring of controllers

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US14/830,401 US11064009B2 (en) 2015-08-19 2015-08-19 Augmented reality-based wiring, commissioning and monitoring of controllers
US14/830,401 2015-08-19

Publications (1)

Publication Number Publication Date
WO2017031033A1 true WO2017031033A1 (en) 2017-02-23

Family

ID=58051379

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2016/046978 WO2017031033A1 (en) 2015-08-19 2016-08-15 Augmented reality-based wiring, commissioning and monitoring of controllers

Country Status (3)

Country Link
US (1) US11064009B2 (en)
EP (1) EP3338240A4 (en)
WO (1) WO2017031033A1 (en)

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019043500A1 (en) * 2017-08-30 2019-03-07 Belimo Holding Sa Automatic assignment between flow control devices, sensor devices and control devices in an hvac application
WO2020120973A3 (en) * 2018-12-12 2020-07-23 Pentair Plc Predictive and preventative maintenance systems for connected water devices
USD899495S1 (en) 2019-03-22 2020-10-20 Lucyd Ltd. Smart glasses
USD899499S1 (en) 2019-03-22 2020-10-20 Lucyd Ltd. Smart glasses
USD899500S1 (en) 2019-03-22 2020-10-20 Lucyd Ltd. Smart glasses
USD899498S1 (en) 2019-03-22 2020-10-20 Lucyd Ltd. Smart glasses
USD899493S1 (en) 2019-03-22 2020-10-20 Lucyd Ltd. Smart glasses
USD899497S1 (en) 2019-03-22 2020-10-20 Lucyd Ltd. Smart glasses
USD899496S1 (en) 2019-03-22 2020-10-20 Lucyd Ltd. Smart glasses
USD899494S1 (en) 2019-03-22 2020-10-20 Lucyd Ltd. Smart glasses
USD900205S1 (en) 2019-03-22 2020-10-27 Lucyd Ltd. Smart glasses
USD900206S1 (en) 2019-03-22 2020-10-27 Lucyd Ltd. Smart glasses
USD900203S1 (en) 2019-03-22 2020-10-27 Lucyd Ltd. Smart glasses
USD900204S1 (en) 2019-03-22 2020-10-27 Lucyd Ltd. Smart glasses
USD900920S1 (en) 2019-03-22 2020-11-03 Lucyd Ltd. Smart glasses
US10908419B2 (en) 2018-06-28 2021-02-02 Lucyd Ltd. Smartglasses and methods and systems for using artificial intelligence to control mobile devices used for displaying and presenting tasks and applications and enhancing presentation and display of augmented reality information
US11282523B2 (en) 2020-03-25 2022-03-22 Lucyd Ltd Voice assistant management
USD954136S1 (en) 2019-12-12 2022-06-07 Lucyd Ltd. Smartglasses having pivot connector hinges
USD954137S1 (en) 2019-12-19 2022-06-07 Lucyd Ltd. Flat connector hinges for smartglasses temples
USD954135S1 (en) 2019-12-12 2022-06-07 Lucyd Ltd. Round smartglasses having flat connector hinges
USD955467S1 (en) 2019-12-12 2022-06-21 Lucyd Ltd. Sport smartglasses having flat connector hinges
USD958234S1 (en) 2019-12-12 2022-07-19 Lucyd Ltd. Round smartglasses having pivot connector hinges
USD974456S1 (en) 2019-12-19 2023-01-03 Lucyd Ltd. Pivot hinges and smartglasses temples

Families Citing this family (71)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9411327B2 (en) 2012-08-27 2016-08-09 Johnson Controls Technology Company Systems and methods for classifying data in building automation systems
EP3274932A1 (en) * 2015-03-24 2018-01-31 Carrier Corporation Integrated system for sales, installation, and maintenance of building systems
US10534326B2 (en) 2015-10-21 2020-01-14 Johnson Controls Technology Company Building automation system with integrated building information model
US11268732B2 (en) 2016-01-22 2022-03-08 Johnson Controls Technology Company Building energy management system with energy analytics
US11947785B2 (en) 2016-01-22 2024-04-02 Johnson Controls Technology Company Building system with a building graph
WO2017173167A1 (en) 2016-03-31 2017-10-05 Johnson Controls Technology Company Hvac device registration in a distributed building management system
US11226598B2 (en) 2016-05-04 2022-01-18 Johnson Controls Technology Company Building system with user presentation composition based on building context
US10417451B2 (en) 2017-09-27 2019-09-17 Johnson Controls Technology Company Building system with smart entity personal identifying information (PII) masking
US10505756B2 (en) 2017-02-10 2019-12-10 Johnson Controls Technology Company Building management system with space graphs
US11774920B2 (en) 2016-05-04 2023-10-03 Johnson Controls Technology Company Building system with user presentation composition based on building context
US20180061269A1 (en) * 2016-09-01 2018-03-01 Honeywell International Inc. Control and safety system maintenance training simulator
US10907853B2 (en) 2016-11-11 2021-02-02 Johnson Controls Technology Company Systems and methods for providing custom applications for HVAC systems
US10275943B2 (en) * 2016-12-13 2019-04-30 Verizon Patent And Licensing Inc. Providing real-time sensor based information via an augmented reality application
US10684033B2 (en) 2017-01-06 2020-06-16 Johnson Controls Technology Company HVAC system with automated device pairing
US11900287B2 (en) 2017-05-25 2024-02-13 Johnson Controls Tyco IP Holdings LLP Model predictive maintenance system with budgetary constraints
US20190361412A1 (en) 2017-02-10 2019-11-28 Johnson Controls Technology Company Building smart entity system with agent based data ingestion and entity creation using time series data
US11280509B2 (en) 2017-07-17 2022-03-22 Johnson Controls Technology Company Systems and methods for agent based building simulation for optimal control
US10854194B2 (en) 2017-02-10 2020-12-01 Johnson Controls Technology Company Building system with digital twin based data ingestion and processing
US11360447B2 (en) 2017-02-10 2022-06-14 Johnson Controls Technology Company Building smart entity system with agent based communication and control
US10515098B2 (en) 2017-02-10 2019-12-24 Johnson Controls Technology Company Building management smart entity creation and maintenance using time series data
US11307538B2 (en) 2017-02-10 2022-04-19 Johnson Controls Technology Company Web services platform with cloud-eased feedback control
US11764991B2 (en) 2017-02-10 2023-09-19 Johnson Controls Technology Company Building management system with identity management
US10169486B2 (en) 2017-02-10 2019-01-01 Johnson Controls Technology Company Building management system with timeseries processing
WO2018175912A1 (en) 2017-03-24 2018-09-27 Johnson Controls Technology Company Building management system with dynamic channel communication
US11327737B2 (en) 2017-04-21 2022-05-10 Johnson Controls Tyco IP Holdings LLP Building management system with cloud management of gateway configurations
US10788229B2 (en) 2017-05-10 2020-09-29 Johnson Controls Technology Company Building management system with a distributed blockchain database
US11022947B2 (en) 2017-06-07 2021-06-01 Johnson Controls Technology Company Building energy optimization system with economic load demand response (ELDR) optimization and ELDR user interfaces
US20180357922A1 (en) 2017-06-08 2018-12-13 Honeywell International Inc. Apparatus and method for assessing and tracking user competency in augmented/virtual reality-based training in industrial automation systems and other systems
WO2018232147A1 (en) 2017-06-15 2018-12-20 Johnson Controls Technology Company Building management system with artificial intelligence for unified agent based control of building subsystems
EP3655824A1 (en) 2017-07-21 2020-05-27 Johnson Controls Technology Company Building management system with dynamic work order generation with adaptive diagnostic task details
US10648692B2 (en) 2017-07-27 2020-05-12 Johnson Controls Technology Company Building management system with multi-dimensional analysis of building energy and equipment performance
US11768826B2 (en) 2017-09-27 2023-09-26 Johnson Controls Tyco IP Holdings LLP Web services for creation and maintenance of smart entities for connected devices
US11314788B2 (en) 2017-09-27 2022-04-26 Johnson Controls Tyco IP Holdings LLP Smart entity management for building management systems
US10962945B2 (en) 2017-09-27 2021-03-30 Johnson Controls Technology Company Building management system with integration of data into smart entities
US20190095821A1 (en) 2017-09-27 2019-03-28 Johnson Controls Technology Company Building risk analysis system with expiry time prediction for threats
US11281169B2 (en) 2017-11-15 2022-03-22 Johnson Controls Tyco IP Holdings LLP Building management system with point virtualization for online meters
US10809682B2 (en) 2017-11-15 2020-10-20 Johnson Controls Technology Company Building management system with optimized processing of building system data
US11127235B2 (en) 2017-11-22 2021-09-21 Johnson Controls Tyco IP Holdings LLP Building campus with integrated smart environment
US10760815B2 (en) * 2017-12-19 2020-09-01 Honeywell International Inc. Building system commissioning using mixed reality
US11874011B2 (en) * 2018-01-19 2024-01-16 Schneider Electric Buildings Americas, Inc. Intelligent commissioning of building automation controllers
US11954713B2 (en) 2018-03-13 2024-04-09 Johnson Controls Tyco IP Holdings LLP Variable refrigerant flow system with electricity consumption apportionment
US11016648B2 (en) 2018-10-30 2021-05-25 Johnson Controls Technology Company Systems and methods for entity visualization and management with an entity node editor
US20200162280A1 (en) 2018-11-19 2020-05-21 Johnson Controls Technology Company Building system with performance identification through equipment exercising and entity relationships
DE102018132921A1 (en) * 2018-12-19 2020-06-25 Endress+Hauser SE+Co. KG Method for operating a field device of automation technology in an augmented reality / mixed reality environment
US11768977B2 (en) * 2019-01-10 2023-09-26 Johnson Controls Tyco IP Holdings LLP Systems and methods for installing and wiring building equipment
US11436567B2 (en) 2019-01-18 2022-09-06 Johnson Controls Tyco IP Holdings LLP Conference room management system
US10788798B2 (en) 2019-01-28 2020-09-29 Johnson Controls Technology Company Building management system with hybrid edge-cloud processing
US11894944B2 (en) 2019-12-31 2024-02-06 Johnson Controls Tyco IP Holdings LLP Building data platform with an enrichment loop
US11356292B2 (en) 2019-12-31 2022-06-07 Johnson Controls Tyco IP Holdings LLP Building data platform with graph based capabilities
US11287155B2 (en) 2020-02-11 2022-03-29 Honeywell International Inc. HVAC system configuration with automatic parameter generation
US11237534B2 (en) 2020-02-11 2022-02-01 Honeywell International Inc. Managing certificates in a building management system
US11526976B2 (en) 2020-02-11 2022-12-13 Honeywell International Inc. Using augmented reality to assist in device installation
US11537386B2 (en) 2020-04-06 2022-12-27 Johnson Controls Tyco IP Holdings LLP Building system with dynamic configuration of network resources for 5G networks
US11874809B2 (en) 2020-06-08 2024-01-16 Johnson Controls Tyco IP Holdings LLP Building system with naming schema encoding entity type and entity relationships
EP3945697A1 (en) 2020-07-30 2022-02-02 Carrier Corporation Method and system for retrieving building automation system controller information using near field communication
WO2022061858A1 (en) * 2020-09-28 2022-03-31 Orange Method for assisting a first user provided with a terminal at a scene
US11954154B2 (en) 2020-09-30 2024-04-09 Johnson Controls Tyco IP Holdings LLP Building management system with semantic model integration
US11397773B2 (en) 2020-09-30 2022-07-26 Johnson Controls Tyco IP Holdings LLP Building management system with semantic model integration
US11847310B2 (en) 2020-10-09 2023-12-19 Honeywell International Inc. System and method for auto binding graphics to components in a building management system
US20220138492A1 (en) 2020-10-30 2022-05-05 Johnson Controls Technology Company Data preprocessing and refinement tool
US11480358B2 (en) 2021-02-25 2022-10-25 Synapse Wireless, Inc. Machine learning systems for modeling and balancing the activity of air quality devices in industrial applications
WO2022197964A1 (en) 2021-03-17 2022-09-22 Johnson Controls Tyco IP Holdings LLP Systems and methods for determining equipment energy waste
CN113341568A (en) * 2021-06-04 2021-09-03 深圳市前海合航科技有限公司 Intelligent wearable and air purification combined device
US11769066B2 (en) 2021-11-17 2023-09-26 Johnson Controls Tyco IP Holdings LLP Building data platform with digital twin triggers and actions
US11899723B2 (en) 2021-06-22 2024-02-13 Johnson Controls Tyco IP Holdings LLP Building data platform with context based twin function processing
US11651376B2 (en) * 2021-07-22 2023-05-16 Bank Of America Corporation Smart glasses based detection of ATM fraud
WO2023086838A1 (en) * 2021-11-11 2023-05-19 Preddio Technologies Inc. Contact identification attachment for electrical connectors
US11796974B2 (en) 2021-11-16 2023-10-24 Johnson Controls Tyco IP Holdings LLP Building data platform with schema extensibility for properties and tags of a digital twin
US11934966B2 (en) 2021-11-17 2024-03-19 Johnson Controls Tyco IP Holdings LLP Building data platform with digital twin inferences
US11704311B2 (en) 2021-11-24 2023-07-18 Johnson Controls Tyco IP Holdings LLP Building data platform with a distributed digital twin
US11714930B2 (en) 2021-11-29 2023-08-01 Johnson Controls Tyco IP Holdings LLP Building data platform with digital twin based inferences and predictions for a graphical building model

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110115816A1 (en) 2009-11-16 2011-05-19 Alliance For Sustainable Energy, Llc. Augmented reality building operations tool
US20110164163A1 (en) 2010-01-05 2011-07-07 Apple Inc. Synchronized, interactive augmented reality displays for multifunction devices
US20130120450A1 (en) * 2011-11-14 2013-05-16 Ig Jae Kim Method and apparatus for providing augmented reality tour platform service inside building by using wireless communication device
US20130169681A1 (en) * 2011-06-29 2013-07-04 Honeywell International Inc. Systems and methods for presenting building information
US20140218398A1 (en) * 2011-04-07 2014-08-07 International Business Machines Corporation Systems and methods for managing computing systems utilizing augmented reality
US20150193982A1 (en) * 2014-01-03 2015-07-09 Google Inc. Augmented reality overlays using position and orientation to facilitate interactions between electronic devices

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5594908A (en) * 1989-12-27 1997-01-14 Hyatt; Gilbert P. Computer system having a serial keyboard, a serial display, and a dynamic memory with memory refresh
US8347957B2 (en) * 2009-07-14 2013-01-08 Halliburton Energy Services, Inc. System and method for servicing a wellbore
EP2302470A3 (en) * 2009-09-29 2014-06-11 Honeywell International Inc. Systems and methods for configuring a building management system
US8681178B1 (en) 2010-11-02 2014-03-25 Google Inc. Showing uncertainty in an augmented reality application
US8890896B1 (en) 2010-11-02 2014-11-18 Google Inc. Image recognition in an augmented reality application
US20120249588A1 (en) 2011-03-22 2012-10-04 Panduit Corp. Augmented Reality Data Center Visualization
JP5810696B2 (en) * 2011-07-12 2015-11-11 富士通株式会社 Variable dispersion compensation device, optical receiver, and variable dispersion compensation control method
US9355648B2 (en) * 2011-11-09 2016-05-31 Nec Corporation Voice input/output device, method and programme for preventing howling
JP5982805B2 (en) 2011-12-13 2016-08-31 株式会社大林組 Building management system and building management method
US10013531B2 (en) 2012-12-20 2018-07-03 Accenture Global Services Limited Context based augmented reality

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110115816A1 (en) 2009-11-16 2011-05-19 Alliance For Sustainable Energy, Llc. Augmented reality building operations tool
US20110164163A1 (en) 2010-01-05 2011-07-07 Apple Inc. Synchronized, interactive augmented reality displays for multifunction devices
US20140218398A1 (en) * 2011-04-07 2014-08-07 International Business Machines Corporation Systems and methods for managing computing systems utilizing augmented reality
US20130169681A1 (en) * 2011-06-29 2013-07-04 Honeywell International Inc. Systems and methods for presenting building information
US20130120450A1 (en) * 2011-11-14 2013-05-16 Ig Jae Kim Method and apparatus for providing augmented reality tour platform service inside building by using wireless communication device
US20150193982A1 (en) * 2014-01-03 2015-07-09 Google Inc. Augmented reality overlays using position and orientation to facilitate interactions between electronic devices

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3338240A4

Cited By (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019043500A1 (en) * 2017-08-30 2019-03-07 Belimo Holding Sa Automatic assignment between flow control devices, sensor devices and control devices in an hvac application
US11391483B2 (en) 2017-08-30 2022-07-19 Belimo Holding Sa Automatic assignment between flow control devices, sensor devices and control devices in an HVAC application
US10908419B2 (en) 2018-06-28 2021-02-02 Lucyd Ltd. Smartglasses and methods and systems for using artificial intelligence to control mobile devices used for displaying and presenting tasks and applications and enhancing presentation and display of augmented reality information
WO2020120973A3 (en) * 2018-12-12 2020-07-23 Pentair Plc Predictive and preventative maintenance systems for connected water devices
USD900203S1 (en) 2019-03-22 2020-10-27 Lucyd Ltd. Smart glasses
USD900204S1 (en) 2019-03-22 2020-10-27 Lucyd Ltd. Smart glasses
USD899493S1 (en) 2019-03-22 2020-10-20 Lucyd Ltd. Smart glasses
USD899497S1 (en) 2019-03-22 2020-10-20 Lucyd Ltd. Smart glasses
USD899496S1 (en) 2019-03-22 2020-10-20 Lucyd Ltd. Smart glasses
USD899494S1 (en) 2019-03-22 2020-10-20 Lucyd Ltd. Smart glasses
USD900205S1 (en) 2019-03-22 2020-10-27 Lucyd Ltd. Smart glasses
USD900206S1 (en) 2019-03-22 2020-10-27 Lucyd Ltd. Smart glasses
USD899500S1 (en) 2019-03-22 2020-10-20 Lucyd Ltd. Smart glasses
USD899498S1 (en) 2019-03-22 2020-10-20 Lucyd Ltd. Smart glasses
USD900920S1 (en) 2019-03-22 2020-11-03 Lucyd Ltd. Smart glasses
USD899499S1 (en) 2019-03-22 2020-10-20 Lucyd Ltd. Smart glasses
USD899495S1 (en) 2019-03-22 2020-10-20 Lucyd Ltd. Smart glasses
USD954136S1 (en) 2019-12-12 2022-06-07 Lucyd Ltd. Smartglasses having pivot connector hinges
USD954135S1 (en) 2019-12-12 2022-06-07 Lucyd Ltd. Round smartglasses having flat connector hinges
USD955467S1 (en) 2019-12-12 2022-06-21 Lucyd Ltd. Sport smartglasses having flat connector hinges
USD958234S1 (en) 2019-12-12 2022-07-19 Lucyd Ltd. Round smartglasses having pivot connector hinges
USD954137S1 (en) 2019-12-19 2022-06-07 Lucyd Ltd. Flat connector hinges for smartglasses temples
USD974456S1 (en) 2019-12-19 2023-01-03 Lucyd Ltd. Pivot hinges and smartglasses temples
US11282523B2 (en) 2020-03-25 2022-03-22 Lucyd Ltd Voice assistant management

Also Published As

Publication number Publication date
EP3338240A4 (en) 2019-05-15
EP3338240A1 (en) 2018-06-27
US11064009B2 (en) 2021-07-13
US20170053441A1 (en) 2017-02-23

Similar Documents

Publication Publication Date Title
US11064009B2 (en) Augmented reality-based wiring, commissioning and monitoring of controllers
US20190332514A1 (en) Mobile application interactive user interface for a remote computing device monitoring a test device
US10594555B2 (en) Cloud-enabled testing of control systems
US8830267B2 (en) Augmented reality building operations tool
EP3352033B1 (en) Maintenance operation support device, maintenance operation support method, and non-transitory computer readable storage medium
US10269235B2 (en) System and method to assist building automation system end user based on alarm parameters
US10534354B2 (en) Device maintenance apparatus, device maintenance system, device maintenance method, device maintenance program, and storage medium
EP2966617A1 (en) Image data generating device, portable terminal device, and portable control device
CN105446178A (en) Commissioning hvac device with optical communication
KR101779827B1 (en) A system and a computer readable storage medium for remote monitoring and controling of fabrication facility
EP3639097A1 (en) Cloud-enabled testing of control systems
US20180365789A1 (en) Device management apparatus, device management method, and non-transitory computer readable storage medium
US10001430B2 (en) Environmental conditioning unit testing
JP7020310B2 (en) Test information management device, test information management method, test information management program, and recording medium
JP6795080B2 (en) Maintenance work support device, maintenance work support method, maintenance work support program and recording medium
US10447769B2 (en) Enhanced time-series data streams for transmission to mobile devices
JP2019197449A (en) Apparatus management system and tag name verification method
CN112035223B (en) Workflow of a device
US11844000B2 (en) Information processing system, information processing method, mobile device, and computer-readable recording medium
JP2015185083A (en) Apparatus management device and apparatus management method
CN114697613A (en) Apparatus, method and recording medium
CN116028316A (en) RPA office flow visualization method and related equipment thereof
JP2021018589A (en) Controller, display unit and equipment management system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16837633

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE