DE102006016760A1 - Procedures for testing BacNet facilities for compliance, interoperability and performance - Google Patents

Abstract

A procedure for the automated testing of building automation systems for conformity to the BACnet standard, a procedure for the automated testing of building automation systems for interoperability according to the BACnet standard and a procedure for the automated testing of building automation systems in a network according to the BACnet standard for performance are presented.

Description

The The invention relates to a method for the automated testing of Building automation facilities on conformity to the BACnet standard, a procedure for automated testing of Building automation facilities according to the BACnet standard on interoperability as well as a method for automated exam of building automation equipment in a network according to the BACnet standard on performance.

The inventive method are in the attached claims and are described in more detail below.

1 Introduction

1.1. objective

In the building automation (GA) big Real estate such as major airports can the use of the internationally standardized communication protocol BACnet (ANSI / ASHRAE 135-2004 or DIN EN ISO 16484-5) - below briefly called "BACnet" - as a communication standard lead to more competition among providers of GA facilities. to Assessment of BACnet suitability is a comprehensive specification to develop, which fully considers all optional possibilities of the communication protocol and the high demands e.g. a major airport. The specifications bear continue to fully meet the specifications set with the to be tested GA manufacturers and drives the further development partially at the device level technically not yet implemented BACnet functions ahead.

For now are tests for conformity to the BACnet standard, interoperability tests and performance investigations only possible in manual and functionally limited form.

Of the Effort for the implementation Manual testing is extreme due to the variety of possible BACnet functionalities time and cost intensive. This represents a more effective market penetration BACnet currently in the way.

by virtue of the high demands on the functionality in the property and the currently limited testing options, is competition over BACnet between several GA brands At high functional level hardly possible at the moment, because the required functionalities are not completely and auditable with reasonable effort are.

Around reduce this effort and carry out BACnet test series To rationalize, the inventive methods have been developed. These Allow procedure In future, every operator will be tested and approved for BACnet devices respective properties and project-specific requirements. By automating the testing, the review of devices becomes different Manufacturer in a reasonable time frame with manageable costs.

First This also becomes an efficient method for the extensive Certification of BACnet components created.

Farther allows flexibility and parameterizability of the test procedures the application to projects any size and different functional Requirement.

First Thus, an open competition is tailored with these test methods to the respective user-specific requirements with a any number of manufacturers.

The Procedure for the ones to be carried out Test series are divided into the areas:

1.1.1. Reading in the device properties

in the As part of the process, the rational creation of the PICS (Protocol Implementation Conformance Statement) document in electronic form (EPICS, electronic PICS) to prepare the test plans. This automatically automates the properties of the test to be tested equipment read from these online and henceforth serve as the basis for the test database.

The EPICS fulfilled thus according to standard ISO 9646 the function of the documents PICS and PIXIT in electronic form.

1.1.2. conformance testing

at the conformity checks are carried out checking if a device Compliant with the standard. In this case, both BACnet standards and standards are standard additional understand user-specific guidelines.

at The examination on conformity to the BACnet standard it is verified that a BACnet device is exactly to the communication specified in the BACnet standard as well as the specified device behavior (for example, if invalid Telegrams, etc.) stops.

1.1.3. interoperability testing

In interoperability tests the correct interaction based on the BACnet-Nor as well as the user-specific guidelines of different components from different manufacturers.

The execution of interoperability checks the previous successful exam on conformity ahead.

1.1.4. performance tests

at the performance reviews are the offered BACnet components in terms of data throughput, Response times and network bandwidth used measured between later Installation location over the network route used up to the specified communication partners checked and documented.

The execution the performance tests sets the successful exam on interoperability ahead.

2. Standards and test specifications

• • DIN EN ISO 16484-5
• • ANSI/ASHRAE 135-2004
• • ANSI/ASHRAE 135.1 + Addendum-A (zukünftige Norm DIN EN ISO 16484-6 Testing Conformance to BACnet)
• • ISO 9646 Teil 1–7 (Testverfahren und Methoden zur Konformitätsprüfung, ohne Betrachtung von Performance oder Fehlerverhalten und dient als Grundlage für 135.1 sowie die durchzuführenden Testreihen.)
• • ANSI/IEEE 829 (Dokumentformat für die Darstellung von Testergebnissen)
• • BTL-specified tests (In diesem Dokument werden zusätzliche von der BMA spezifizierte Tests beschrieben.)
• • Anwenderspezifische Richtlinien
• • Anwenderspezifische Netzwerkspezifikationen

The following standards and test specifications are the basis for carrying out the tests:

• • DIN EN ISO 16484-5
• • ANSI / ASHRAE 135-2004
• • ANSI / ASHRAE 135.1 + Addendum-A (future standard DIN EN ISO 16484-6 Testing Conformance to BACnet)
• • ISO 9646 Part 1-7 (Test methods and methods of conformance testing, without consideration of performance or error behavior, and serves as the basis for 135.1 as well as the series of tests to be performed.)
• • ANSI / IEEE 829 (document format for displaying test results)
• • BTL-specified tests (This document describes additional tests specified by the BMA.)
• • User-specific guidelines
• • User-specific network specifications

3. Current situation, State of the art

automated Tests (i.e., methods using software-based Tools) for conformity, interoperability and performance in terms of BACnet are state of the art not possible.

3.1. conformance testing

The American Association BMA (BACnet Manufacturers Association, www.bacnetassociation.org) operates a test lab by the name of BTL (BACnet Testing Laboratories) currently based in Boston. In Manual BACnet test series have already been carried out in this test laboratory. The BMA has been preparing test plans based on the standard for several years (ANSI / ASHRAE 135-2004 / BACnet or DIN EN ISO 16484-5) the implementation of Allow tests. However, these tests are currently only in manual Form performed.

The inventive method allow automated testing on conformity to the device profiles of the standard and in addition to the user-specific Conditions.

3.2. interoperability testing

Tests on interoperability different devices different manufacturers are in the US and Europe in Form of so-called "Plugfests" ("plug festivals") organized. there Becomes functions of the standard, which are implemented by manufacturers new in devices were tested against existing implementations.

The Investigation on interoperability especially considering the user-specific requirements with regard to the quantity structure of the supported object types, Telegrams etc. as well as special BACnet functionality in Europe or in the US.

3.3. performance tests

The inventive method allow the measurement of the network bandwidth used, the response times as well as the achievable data throughput of the test units.

4. Description the procedure

The inventive method to carry out BACnet test series are divided into subareas described below on.

4.1. Specification of test preparations

equipment manufacturers put to the test Test units ready. The specifications for the test preparation - below short "test specifications" called - include the user specific requirements as well as the definition of the settings of the test units to be used.

When Test unit is understood to be a single or a group of several BACnet devices, which a required technical building Fulfill function should and conformity to be tested in the implementation of the BACnet standard.

A test specifications ("GA specifications") for the preparation of the test units by the manufacturer is provided by the operator of the property provides.

The for the Conformity test relevant standards and basics will be announced to the manufacturers. The GA specifications in the respective current version becomes the manufacturers likewise to disposal posed.

Of the Manufacturer prepares its test unit according to the requirements before (programming / parameterization) and transfers this to the operator the property.

Together with the test unit passes the manufacturer a documentation of the operating steps as well as a device Description in standardized format "EPICS" (electronic PICS cf. ISO 9646).

Based the BACnet functions described in this document are performed exam on conformity. The manufacturer thus announces the degree of its BACnet implementation with this document.

4.2. Specification of Test sequences

Based The standard ANSI / ASHRAE 135.1 defines the test series to be carried out.

The Norm already describes the for the exam on conformity required BACnet functions.

The for the exam on conformity required BACnet functions are based on the device profiles the BACnet standard and the user-specific guidelines.

4.3. Elaboration of a test software

For the implementation of Tests and to ensure the reproducibility of test results An automated test software is created to check for compliance. This test software allows the automated testing and documentation of the Test results based on the o.g. Specifications, standards and test specifications.

4.4. Construction of a test laboratory

in the Framework of the Tests are the equipment and the required software and hardware tools to establish a test laboratory and documented.

For the implementation of Conformity testing is each test unit is individually connected to the test software.

For the implementation of interoperability testing All test units will work together in any network switched together.

planning and construction of this test network, which is the user-specific Considers network policies, are also part of the test procedures.

For this counting in particular the planning of the needed equipment and test tools as well as the definition of IP addresses of the devices, definition of device name and device addresses.

4.5. Planning of conformity tests

to exam on conformity A test plan is created from which the BACnet services to be tested and BACnet objects including the user-requested optional Properties and the application-specific required write / read access emerge.

These are specified within an input mask and in the test database entered as default.

The for the Conformity test relevant test steps are based on ISO 9646, which generally describes the conformity assessment procedures.

in the Standard 135.1 are the ones for the BACnet exams specified test steps in positive and negative tests.

In addition to these standards is still the GA specifications BACnet in each of the current Version Basis of a release.

4.6. consistency check

The The manufacturer's EPICS file is automatically optimized in terms of content checked. This will be missing information or recognizable by the software incorrect entries issued in an error list and denied the adoption of EPICS in the database.

4.7. Carrying out of conformance testing

Based the test plans become the test units with the help of the automated procedure the manufacturer individually for conformity to the BACnet standard and the user-specific Guidelines tested.

The Software is able to store the test plans in a database based on the test plans and the respective EPICS of the device perform the tests automatically as well as to document the test results electronically.

Farther All test steps are documented electronically, so that these for one Reproducibility called again and understood at any time can be.

4.7.1. objective

To Completion of the conformity test it should be per device a central list (expected in a spreadsheet format) give an overview of all both manually and automatically tested properties.

4.7.2. EPICS exam

in the The first test step is the manufacturer's information in the EPICS document automatically with the user specified in the test database Requirements compared.

Not supported Functions, objects and properties of the test unit are considered negative test results evaluated and automatically stored in the database.

in the The second test step is a so-called EPICS Consistency Test 135.1 performed. The exam steps to the EPICS consistency check are defined in 135.1 and write i.a. before that, an exam like that must be carried out that all functionalities described in EPICS also indeed are present in the test unit.

Gives a manufacturer e.g. in his EPICS, the BIBB (interoperability module) DM-DOB-A support and this is not to be found in the test unit (criterion: readout the property Protocol Services Supported from the device object), This is an inconsistency between EPICS document and device implementation (or device configuration).

These Inconsistencies are documented in the checklist and rated as negative test results.

This Test step is performed automatically. The EPICS file provided by the manufacturer is through the testing software as template, additionally a reading out of the test unit and a subsequent software-related Comparison to set and actual state of the functions. inconsistencies between EPICS and test unit are documented.

4.7.3. conformance test

in the third test step is based on the implemented in the test unit BACnet features an automatic generation of the test plan in the test software.

For this serve those described in the standards for testing individual test steps.

Not supported Functions, objects and properties of the test unit are considered negative test results evaluated and automatically stored in the database.

With Help of the test software will be the positive and negative test results The individual test steps are automatically checked and documented.

The test results are stored in a database and can be evaluated in Office applications be exported.

4.8. Evaluation of the conformity test

The Results of all tests be in a coherent Report compiled. Deviations from the underlying Standards and guidelines are documented in this report. In A builder's meeting, these results in the form of a presentation summarized and presented.

4.9. Planning interoperability tests

For the rating interoperability become test cases with which the evaluation can be carried out, whether and in which Scope of several BACnet devices work together correctly.

10.4. Carrying out of interoperability testing

Based the test cases all devices or Test units in the network network checked for correct interaction.

The Test steps are performed using test tools (network analyzer, etc.) are observed and evaluated.

The while the implementation of the tests recorded using the test tools log files archived by the test personnel and if necessary documented.

These Enable log files a later assessment of possible problem cases and serve as proof of the test.

11.4. Planning of performance tests

The Performance studies are carried out both in the test laboratory and also in the user-specific IT network.

To become test cases worked out a conclusion on the expected performance of the devices in terms of network bandwidth, Allow reaction times as well as overall throughput.

12.4. Carrying out of performance tests

The Performance tests are first independently from the user-specific IT network performed in the test laboratory.

there becomes every single device an investigation on reaction times initially without consideration of the entire Network network subjected.

In In another test step, the network reaction time is between the planned installation location of the device and the location of the communication partners.

in the following test step, the test units become network side at the installation site the network route used up to the specified communication partners connected and measured the performance.

All Test steps become later Use and assessment documented and archived.

13.4. Documentation and Evaluation of the test results

The manually and automatically recorded test steps are used for purposes the reproducibility archived and documented.

The Test results are summarized in the form of presentations and can presented as a basis for assessment become.

5th case example a conformity check

in the The following case study for the conformity test uses the term "manufacturer" in the singular. This is representative of Companies using BACnet components to offer.

Of the Manufacturer submits a test unit, which according to EPICS the BACnet function DM-BR-B (Device Management Backup / Restore as provider) supported.

This means that the test unit over must have at least one BACnet file object.

Of the first test step (test on GA guideline) would be thus for fulfilled this single function.

Becomes determined in the second test step that contrary to the statement in EPICS does not have a BACnet file object, so the test is for this Time ended negative. Also the test step one is thus to be considered negative.

in the The third step is now all in 135.1 or the accompanying standards specified tests automatically performed.

there is the device behavior clearly defined in positive and negative tests.

A Single function (exam of the individual BACnet BIBBs) can only be considered as passed if all the individual test steps were successfully completed on the basis of 135.1.

in the Ideally fulfilled a test unit of a manufacturer all the requirements of the GA specifications, the standards 135.1 and 16484-5 as well as the accompanying extensions completely.

5.1. Representation of the automated test run based on ANSI / ASHRAE 135.1

in the Standard 135.1 are for Each BIBB to be tested (BACnet Interoperability Building Block, Interoperability block) various individual steps are deposited, which are carried out as a whole need to, um the conformity for this BIBB check.

The test steps include both positive tests, i. the test software sends a valid telegram with correct data content and expects from the examinee a valid result, which either automated over the BACnet protocol e.g. through a list of valid return values or through e.g. Reading a value e.g. on an operator display (current Room temperature, etc.) is verified by the operator of the test software.

In the negative tests are syntactically correct data telegrams but with an invalid Data content to the examinee Posted. Subsequently will verify if the examinee behaves correctly according to the specifications of the standard, e.g. returns a correct error code.

When In this case, the test steps for testing the ReadPropertyServiceExecutionTests from the application field Serve execution tests. These test steps check compliance with the conformity in the area of the BIBB DS-RP-B and serves here as an example for all BACnet BIBBs.

5.1.1. Positive Tests ReadPropertyExecution

In this area are two individual tests spe zifiziert:

Reading the size of an array

Here is checked, whether the size of a Arrays processed correctly.

Reading a single element of an array

Here is checked, whether reading a single element of the array correctly handles becomes.

5.1.2. Negative tests ReadPropertyExecution

In Four individual tests are specified in this area:

Reading Non-Array Properties with an Array Index

Here is checked, whether the examinee a valid one Error code returns when attempting to use an item (property) Specify an array index to read, with the referenced Property is not an array.

Reading Array Properties with an array index that is out of range

Here is checked, whether the examinee a valid one Error code returns when attempting to make an array with the indication an array index outside of the array area.

Reading to Unknown Object

Here is checked, whether the examinee a valid one Error code returns when attempting an object that does not exist in the DUT to read.

Reading to Unknown Property

Here is checked, whether the examinee a valid one Error code returns when attempting to create a non-object Property (Property) read out.

The in 135.1 specified test cases as well as all Test procedures and Result documentation takes place within the procedure as far as possible automatically.

Furthermore In the course of the tests, manual test steps are also carried out. The manual ones Tests relate e.g. on the rating read (by the operator reading) values on a display, mainly in the area of BACnet client functionality. These are also marked as such in 135.1.

5.1.3. result review

Based the error codes or expected results specified in 135.1 the evaluation is made as to whether a candidate is the one specified Single test has passed successfully.

One Single step can only be considered successful if the result set in the range of 135.1 specified results For manual readings, this may be in a specified value range.

6th case study interoperability test

In the interoperability test become exclusive already on conformity tested Devices used.

6.1. objective

There already in the conformity tests the check up correct handling of data telegrams and behavior in case of error were checked the focus on interoperability testing in the investigation Timing behavior of the devices as well as matching application implementations of the individual equipment in the practical test.

Support both equipment mating BIBBs, e.g. one device DS-RP-A and the other device DS-RP-B, this statement says nothing left over that support Quantity structure, So the common intersection of both devices. The established interoperability is the Data throughput supported by both devices.

For example, could a PC based client an automation station (DDC device) under circumstances in extreme cases, with requests such overload that communication due to telegram repetitions can be extremely slow to the fullest Failure of the communication.

The Review and document interoperability tests therefore the votes between different devices to to achieve optimal communication.

Farther the BACnet standard only describes the communication in the ISO / OSI model layers 1 to 7, but not the implementation of the respective BACnet application. Here, e.g. a check up a different processing of values in devices of different Manufacturer, e.g. the processing of decimal places, etc.

6.2. Preparations of the interoperability testing

to Preparation of the interoperability test must first of all devices to be examined be integrated into a common network network, i. from every device to every other device There is a physical and communicative connection.

Examines in each case the combination of the application implementations of two equipment in this composite.

Determination of the test scope:

tested can be only matched BACnet features supported by both devices (common intersection).

A

= Anforderer (Client)

B

= Bereitsteller (Server)

For this, a differentiation is made in the BACnet standard when specifying the BIBBs in A and B functions

A

= Requestor (client)

B

= Provider (server)

provides now a device If a B function is ready (e.g., ReadProperty-B), then the device must be the one to which interoperability checked will, over have the corresponding client function ReadProperty-A.

is in this interoperability area (IOB) no match This area can not be checked for interoperability.

Based the entire list of features supported by both devices is the exam on Interoperability.

6.3. Carrying out the interoperability testing

each single from both devices supported Function is a single test step of interoperability testing.

On Application level is now the provided value or the provided BACnet function (for example, data backup) is read from the provider and documented.

Of the Requester reads this value or performs the provided function at the provider.

6.4. Evaluation of the interoperability test

Complies the value read out was the previously determined result or was the Function successfully completed on both sides, this applies Single step fulfilled successfully.

is in the overall list of IOB for a certain area no match both devices existed or became the exam of the respective single step does not end successfully, so this is Single step as unsuccessful.

These it is the devices to different manufacturers, it is necessary to check and determine which ones Implementation changes are needed to achieve interoperability.

When Example could be one of the two manufacturers increase timing or resource parameters to to adapt to the characteristics of the other communication partner.

The Test series Interoperability tests fulfilled with it the function of determining a decision basis of possible Extensions of the device applications to be taken.

7th case study performance analysis

The The aim of the performance investigations is to test and Documentation of network properties of the IT infrastructure as well the exam potential Influence on the communication of a BACnet network.

As the name implies, it is all about Investigations on resources and timing behavior in connection with the network infrastructure provided by the customer.

7.1. Preparations of the performance tests

For the performance investigations have to first put together suitable tools for measuring the time behavior become. For this serve commercial Tools such as network monitors and analyzers and additionally BACnet-specific tools to be developed, such as i.a. "BACnet ping".

The to use program "BACnet-ping" works analogously to the usual in the IT environment Tool "ping" with the difference that this particular to the for BACnet required, protocol-specific properties (UDP protocol, used Port numbers, etc.) is adapted and extended.

This Tool allows the time measurement of a BACnet telegram with a defined Data content about a defined section of the BACnet network.

7.2. Carrying out the performance tests

All for the exam The definitions to be defined in the performance will be based on the expected Data traffic between two stations (minimum requirement to the Network traffic) in addition of a safety impact in an entire checklist. Furthermore contains the checklist the specification of a desired Minimum requirement for the maximum permissible duration of each individual step.

In different test steps are each different Quantities as well as different access points and sections of the network for following subsequent exam established.

During the exam becomes the set of BACnet telegrams defined in the respective test step with a defined data content from a defined point of the Network to another defined point of the network over a defined Sent section of the network.

The Time required to transport this amount of data will be included logged at both access points to the network and in the checklist documented.

7.3. Evaluation of the performance investigations

Complies the temporal behavior the desired Minimum time requirements, this test step is considered successful checked.

Becomes for the transmission a defined amount of data via a defined distance of the network needs more time than expected, so this test step is considered unsuccessful.

The Interface for the performance tests start from the BACnet telegram, which from the device is sent until the arrival of the telegram at the receiver via the entire way the network route.

in the The scope of the performance investigations will vary with different "telegram loads" (different packet sizes and Package volume) of the network worked to potential effects to be able to infer the performance and overall performance of the system.

by virtue of The results of the performance investigations may be project specific appropriate measures such as. Removal of latencies in the network, selection of other access points, etc. To be defined.

The Performance investigations thus serve as a basis for decision-making for selection and specification of BACnet devices to be used and for Determination of network requirements.

Claims (10)

Method for automated testing of Building automation facilities on conformity to the BACnet standard, in which - with a test unit one associated device Description (EPICS) in electronic standardized format, - user-specific Requirements are recorded in a test database, - the information in the device description and with the requirements defined in the test database to be compared - to consistency check the properties of the test unit read and with the information the device description to be compared - at successful consistency check a test plan defining the individual test steps based on the read-in device Description is generated - the Test steps performed one after the other become; and - the Test results for documentation automated in the test database be deposited. Method according to claim 1, in which it is checked whether to a function listed in the information of the device description The test unit has a BACnet object of type File is. The method of claim 2, wherein to each one Function the test steps specified in the standard (ANSI / ASHRAE 135.1) carried out be sent by sending appropriate data telegrams to the test unit and verify the return values obtained from the test unit. The method of claim 3, wherein the return values with a list of valid ones return values be compared. Method according to claim 3 or 4, in which also syntactically correct data telegrams with invalid data content to the test unit be sent to check whether the test unit has a valid Returns error code. Method for the automated testing of building automation devices according to the BACnet standard for interoperability, in which - the test units to be checked are integrated into a network in a network, - for each two test units of the network, one of which requestor (client) and the other provider ( Server), all of both test units are un supported BACnet functions are tested automatically one after the other. The method of claim 6, wherein the provider supplied data value is read out and documented as well as by the requester is read. The method of claim 6, wherein the provider provided BACnet function is read out and documented as well is executed by the requester at the provider. Method according to one of claims 6 to 8, which after successful conformity check one of the claims 1 to 5 performed becomes. Method for automated testing of building automation devices in a network according to the BACnet standard on performance, at wherein - one checklist is generated in which the for the exam Definitions based on the expected performance Data volume between two test units (stations) plus one Safety margin be specified - for consecutive test steps different amounts of BACnet data telegrams as well set different access points and sections of the network become, - in For each test step, the specified amount of BACnet data telegrams with a defined data content from a defined point of the Network to another defined point of the network via a network defined section of the network is sent, - in the Each test step for transporting the set of BACnet data telegrams Log time required between the defined points on the network and in a checklist is documented.