The Niagara Framework uses the TestNG test framework for executing unit tests within the Niagara Framework. TestNG is a well-known tool in the Java community, and in addition to creating basic unit tests, it supports the following functionality:
Niagara’s gradle-based build system supports co-located test source code (that is, source code and test code are contained in the same module development folder). More information on TestNG can be found in the TestNG Documentation
Create a “srcTest” folder in the module, and add all your test source code there. Here is an example of a module with a test class:
Each test class should extend javax.baja.test.BTestNg and should include the standard Baja code to declare the Type. TestNg will treat each test method as a single test case. A test method is defined by annotating it with the @Test annotation:
package com.acme.myModule.test;
import javax.baja.nre.annotations.NiagaraType;
import javax.baja.sys.Sys;
import javax.baja.sys.Type;
import javax.baja.test.BTestNg;
import com.acme.myModule.BFunctionType;
import org.testng.Assert;
import org.testng.annotations.Test;
@NiagaraType
public class BFunctionTypeTest extends BTestNg
{
/*+ ------------ BEGIN BAJA AUTO GENERATED CODE ------------ +*/
/*@ $com.acme.myModule.test.BFunctionTypeTest(2979906276)1.0$ @*/
/* Generated Mon Mar 28 07:19:53 EDT 2016 by Slot-o-Matic (c) Tridium, Inc. 2012 */
////////////////////////////////////////////////////////////////
// Type
////////////////////////////////////////////////////////////////
@Override
public Type getType() { return TYPE; }
public static final Type TYPE = Sys.loadType(BFunctionTypeTest.class);
/*+ ------------ END BAJA AUTO GENERATED CODE -------------- +*/
@Test
public void addTest()
{
Assert.assertEquals(BFunctionType.make(BFunctionType.ADD), BFunctionType.add);
}
}
There are several assert*() methods available in TestNG to test equality, null, true/false, etc. See the TestNG Javadocs for the complete list.
One big advantage of TestNG is its flexible test configuration options. As with specifying tests, these configurations are also specified with annotations. Setup and teardown methods can be established to run once per method, per class, per test group, or per test suite. The example below shows how to initialize some test object before each test method is run.
@BeforeMethod(alwaysRun=true)
public void setup()
{
myTestObject = new MyTestObject();
myTestObject.setup();
}
@Test
public void testMyObject()
{
Assert.assertTrue(myTestObject.isInitialized());
...
}
private MyTestObject myTestObject;
The javax.baja.test.TestHelper class has several utility methods that can be useful in certain situations. There are methods to get the value of a private variable, set the value of a private variable, and invoke a private method. The following will set the value of a private String variable “somePrivateField” on an instance of MyObject:
MyObject myObj = new MyObject();
TestUtil.setPrivateField(myObj, "somePrivateField", "Some New Value");
There are methods that wait for asynchronous changes. In the example below, execution will wait for a component action that runs asynchronously to complete before continuing:
private void runAction(BComponent component)
{
component.performAsyncAction(BBoolean.FALSE);
TestHelper.waitFor(() -> !component.getAsyncResult());
}
There are methods that assert test results on asynchronous changes. This example asserts that a Tag Dictionary version changes from “1.0” to some other value as a result of the importDictionary action:
myTagDictionary.doImportDictionary(Context.NULL);
assertWillBeTrue(() -> !"1.0".equals(myTagDictionary.getVersion()), 10000, "Waiting for import to finish");
The wait methods have a default timeout of 5000 milliseconds, and there are additional versions of the methods that accept an explicit timeout value.
Other TestHelper methods also have several versions with different arguments, so look at the TestHelper API Bajadocs for specific details.
There should be no changes necessary to the default build.gradle/<moduleName>.gradle scripts to compile unit test cases. If the unit tests require non-Java resources (BOG files, images, input files, etc), gradle will need to be configured to include them in the resulting test jar. For example, if there is an “rc” folder in the “srcTest” folder with the necessary test resources:
tasks.named<Jar>("moduleTestJar") {
from("srcTest") {
include("rc/**")
}
}
Types for unit tests must be declared, but this should be generated automatically when tests are compiled (see below)
There is a “moduleTestJar” gradle task which will build a test module containing all the unit tests for a given module:
gradlew moduleTestJar
Once the module test jar has been generated, the test command is used to run the tests defined for a given module. The test command can take one of three arguments:
Single-method execution is not currently supported. Test output will look something like this:
C:\Users\user\Niagara4.2\tridium\myModule\myModule-rt> test myModule
[TestNG] Running:
Command line suite
===============================================
myModuleTest_FunctionTypeTest
Total tests run: 5, Failures: 0, Skips: 0
===============================================
Output verbosity can be set using the option v:<n>, where n is an integer from 1 to 10. The higher the number, the more output provided by TestNG.
Some tests run code that must be executed in a running station. For example, your code might depend on a running User Service or Niagara Network in order to function properly. Your test class can extend javax.baja.test.BTestNgStation, which creates a basic running station that includes many basic services.
package com.acme.myModule.test;
@NiagaraType
@Test
public class BMyComponentTest extends BTestNgStation
{
//region /*+ ------------ BEGIN BAJA AUTO GENERATED CODE ------------ +*/
//@formatter:off
/*@ $com.acme.myModule.test.BMyComponentTest(2979906276)1.0$ @*/
/* Generated Tue Jan 18 11:02:21 CST 2022 by Slot-o-Matic (c) Tridium, Inc. 2012-2022 */
//region Type
@Override
public Type getType() { return TYPE; }
public static final Type TYPE = Sys.loadType(BMyComponentTest.class);
//endregion Type
//@formatter:on
//endregion /*+ ------------ END BAJA AUTO GENERATED CODE -------------- +*/
public void testMyComponentInRunningStation()
{
BMyComponent myComponent = new BMyComponent();
stationHandler.getStation().add("MyComponent", myComponent);
...
}
}
You can extend the default station to add components or change configurations prior to test station startup by overriding the configureTestStation method. You will typically want to initialize the default station by calling super.configureTestStation. In this example, a Tag Dictionary Service is added to the station.
@Override
protected void configureTestStation(BStation station, String stationName, int webPort, int foxPort)
throws Exception
{
super.configureTestStation(station, stationName, webPort, foxPort);
station.getServices().add("TagDictionaryService", new BTagDictionaryService());
...
}
Alternatively, you can add components to the station using a bog file or and XML station file. Include the file in your test module, then override the makeStationHandler method to load the station components.
@Override
protected void makeStationHandler()
{
stationHandler = createTestStation(BOrd.make("module://myModule/test/rc/test.bog"));
}
If the station definition file contains services configured in the default station, you will need to override the configureTestStation without the call to the super.configureTestStation method to avoid duplicate slot conflicts.
The default test station creates a few basic roles and users in the default Role and User Services. You can add roles and users to these defaults by calling addRole and addUser methods.
@Override
protected void configureTestStation(BStation station, String stationName, int webPort, int foxPort) throws Exception
{
super.configureTestStation(station, stationName, webPort, foxPort);
// Add admin read role and user
addRole(TEST_ADMIN_READ, map(BPermissions.make("rwiR")));
addUser(TEST_ADMIN_READ, TEST_ADMIN_READ);
...
}
private static final String TEST_ADMIN_READ = "TestAdminRead";
If you need restrict the roles and users to only those you configure, you can do this by overriding the makeRoleService and makeUserService methods.
@Override
public BComponent makeRoleService() throws Exception
{
return new BRoleService();
}
@Override
public BComponent makeUserService() throws Exception
{
return new BUserService();
}
@Override
protected void configureTestStation(BStation station, String stationName, int webPort, int foxPort) throws Exception
{
super.configureTestStation(station, stationName, webPort, foxPort);
// Add admin read role and user
addRole(TEST_ADMIN_READ_USER, map(BPermissions.make("rwiR")));
addUser(TEST_ADMIN_READ_USER, TEST_ADMIN_READ_USER);
...
}
private static final String TEST_ADMIN_READ_USER = "TestAdminRead";
Several getter methods are included to return specific services, containers, and other properties associated with the default test station. Review the BTestNgStation API Bajadocs for the complete list of these access methods.
A client connection to the Fox service in the running station can be established by calling the connect method. This can allow you to simulate a Fox client connection (e.g. Workbench connection to a station).
BProxyFoxSession session = connect("TestAdmin", "Test@1234");
The web and search services are disabled by default, since the majority of tests don’t rely on them and having them disabled allows the test station to start faster. They can be enabled by overriding methods.
@Override
protected boolean isWebServiceEnabled()
{
return true;
}
@Override
protected boolean isSearchServiceEnabled()
{
return true;
}
A set of tests may be grouped together with the groups annotation attribute. Groups naming is currently up to the developer. One use of groups is to identify a collection of tests to execute in a Continuous Integration (CI) environment. Note that CI is not provided by Niagara; see this link for more information.
An example of the group attribute is below:
@Test(groups={"ci"})
public void ngTestSimple()
{
Assert.assertEquals(Lexicon.make("test").getText("fee.text"), "Fee");
}
You may declare dependencies between test methods and groups using dependsOn* annotation attributes. For example, if you have a group of tests that should run after other sets, just attach the dependsOnGroups attribute for each method in the group.
If you want to explicitly define a sequence of test method execution, use the priority annotation attribute. The value is a positive integer, and lower priorities will be scheduled first. See the TestNG documentation for additional information.
Important: If you implement groups and also use BeforeClass/AfterClass methods, be sure to attach the alwaysRun=true attribute to the BeforeClass/AfterClass annotations
A set of similar test cases can be parameterized with a data source class that generates input to the test method. Again, the relationship is designated with parameterized annotations. First, you declare a data provider that creates an object array containing test method arguments for for each instance of the test execution. In the example below, the test method takes two arguments, and each entry in the data provider array contains instances of those two argument types.
@DataProvider(name="operation")
public Object[][] createColumnData()
{
return new Object[][] {
{ Integer.valueOf(BFunctionType.ADD), BFunctionType.add },
{ Integer.valueOf(BFunctionType.SUBTRACT), BFunctionType.subtract },
{ Integer.valueOf(BFunctionType.MULTIPLY), BFunctionType.multiply },
{ Integer.valueOf(BFunctionType.DIVIDE), BFunctionType.divide }
};
}
@Test(dataProvider = "operation")
public void testOperation(Integer i, BFunctionType ft)
{
Assert.assertEquals(BFunctionType.make(i.intValue()), ft);
}
Note that data provider argument types must be Java Objects (they cannot be primitives such as boolean on int).
If your code can generate exceptions and you want to test those execution paths, you can tell a test method to expect particular exception types by using the expectedExceptions attribute with a list of exception classes. In the following test, an occurrence of a NullPointerException will successfully pass the test. Any other exception type will fail the test.
@Test(expectedExceptions={java.lang.NullPointerException.class})
public void ngTestException()
throws Exception
{
a = BExportSourceInfo.make(BOrd.make("station:|slot:/a"), BOrd.make("station:|slot:/b"), new BGridToText());
BExportSourceInfo.make("foo:bar");
a.decodeFromString("foo:bar");
}
TestNG will generate XML and HTML reports each time it runs. By default, it creates these in a <niagara.user.home>/reports/testng folder. The HTML report index.html contains detailed information about the test results. There is also a static XML report and an email-able static HTML report. The report location can be changed using the command line option -output:<path>
The Niagara test executable offers several options for tailoring the execution of tests to your needs. The usage and options are outlined below:
usage:
test <target> [target ... target] [testng options]
target:
all
<module>
<module-runtimeProfile>
<module>:<type>
<module>:<type>.<method>
<module>:<type>./regex to match against methods of <type>/
<com.package>.<BTestClass>
<com.package>.<BTestClass>#<method>
<com.package>.<BTestClass>#/regex to match against methods of <BTestClass>/
/<regex to match against the com.package.BTestClass#method format>/
testng options:
-v:<n> Set TestNG output verbosity level (1 - 10)
-output:<path> Set the location for TestNG output
-groups:<a,b,c> Comma-separated list of TestNG group names to test
-excludegroups:<a,b,c> Comma-separated list of TestNG group names to skip
-skipHtmlReport Flag to disable HTML report generation
-generateJunitReport Flag to enable JUnit XML report generation
-benchmark Print the 50 highest duration tests and test suites on exit
-loopCount:<n> Run target(s) in a loop n times (1 - 1000000)
In addition to running your tests with the test command; you can also run them via Gradle. This also enables integration with most IDEs. To run tests via Gradle, run the niagaraTest task:
gradlew :myModule-rt:niagaraTest
As with the test command, you can configure the execution of tests with command-line options for the niagaraTest task:
gradlew help --task :myModule-rt:niagaraTest
...
Options
--excludeGroups list of groups to exclude
--groups list of groups to use
--module module target
--target test target (all, module, module:test, module:test.method)
--type class or class.method target
--verbosity testNG verbosity level (1-10)
In particular, --target takes the same options as -target on the test command.
You can also configure these options in your module’s build file. This is useful if you always want to run tests against a specific group, or exclude certain tests from being run by default:
import com.tridium.gradle.plugins.niagara.task.RunNiagaraTestTask
...
tasks.named<RunNiagaraTestTask>("niagaraTest") {
groups.set(listOf("ci")) // Note this is a list, even if you only pass in one group
excludeGroups.set(listOf("slowTest"))
}
Additionally, running your tests with Gradle enables the use of the JaCoCo code coverage tool. JaCoCo gives you a report of how much of your code was actually exercised by your tests. While this is not in and of itself an indicator of test quality, it can be a useful tool to see areas of your code that your tests do not cover sufficiently.
To run your module’s tests with coverage, run the tests via Gradle:
gradlew :jacocoExample-rt:niagaraTest
This will produce a JaCoCo test coverage data file at jacocoExample-rt/build/jacoco/niagaraTest.exec. This file in and of itself doesn’t do much; you’ll want to generate an HTML report as well. This can be done with the following Gradle command:
gradlew :jacocoExample-rt:jacocoNiagaraTestReport
This will produce an HTML report in jacocoExample-rt/build/reports/jacoco/niagaraTest/html. You can open this report in any web browser to get a view of the test coverage for your module:
