Installing Io Addons

Io is an interesting programming language. I had not planned to learn a new language in 2024. Then, in a quiet moment, I reached for Seven Languages in Seven Weeks by Bruce Tate. It is an older book, published in 2010 and I know some of the languages described there. I like studying programming languages. Playing with basic concepts without the need to finish larger tasks is relaxing. It should be an easy read.

The Io Language
Io "is a dynamic prototype-based programming language." It seems more or less dead since 2010, basically since the book mentions it. Its site states "actively developed until 2008". While there are a few updates from last year, the latest available build for Windows dates to 4th of December 2013, more than ten years ago. Much of Io is written in C and one should be able to build it from source using cmake. While I like the retro character of plain C, similar to old Pascal, I am un-experienced in the C ecosystem. I am using Windows and building C has been proven to be a hassle due to dependencies and compiler inconsistencies. So I went with the provided Windows build.

Io is prototype based, like JavaScript and looks different than your usual C-styled languages. The core library iovm comes with a unit testing framework. The UnitTest is an old-school xUnit, e.g.

FizzBuzzTest := UnitTest clone do (

  setUp := method(
    super(setUp)
    self fizzbuzz := FizzBuzz clone
  )

  // ...

  testFizz := method(
    result := fizzbuzz single(3)
    assertEquals("Fizz", result)
  )

  // ...

  testTo5 := method(
    result := fizzbuzz upto(5)
    assertEquals(list("1", "2", "Fizz", "4", "Buzz"), result)
  )

)

The corresponding Fizz Buzz is

FizzBuzz := Object clone do (

  single := method(n,
    if (n % (3 * 5) == 0, "FizzBuzz",
      if (n % 5 == 0, "Buzz",
        if (n % 3 == 0, "Fizz",
          n asString)))
  )

  upto := method(limit,
    Range 1 to(limit) map(n, single(n))
  )
)

This could be written more concise, but such was my first Io code. I should probably do a Prime Factors, too. It seems that I am obsessed with unit tests, as it is usually the first thing I look for in a new language, e.g. Scheme or Assembly. If you want to know more about Io, read the guide or the second chapter of Seven Languages in Seven Weeks. While the language is dormant, it will change the way you see programs - exactly as Bruce promises.

Io Addons
The Io distribution contain a bunch of extensions collected under the IoLanguage GitHub organisation. Many extensions are wrappers around well tested C libraries, but these "may not be installed on your system already." And this is probably the reason why the Windows distribution only contains 28 out of the 80+ available addons. While I ignore MySQL or graphics library wrappers, basic functions like socket communication or regular expressions would be nice. There is Eerie, the Io package manager, which should be able to deal with compiling addons, but its installation fails on my system - because of a dependency that needs a missing C library (recursion ;-). Then I try to add addons manually. It is the main purpose of this post to explain how it can be done.

Let me start with some general structure of Io addons: Installed or bundled addons are located in %IO_HOME%\​lib\​io\​addons and share the same folder structure as expected by Io's AddonLoader. Assume an addon named Foo, then the following folders and files (could) exist:

• _build: Several folders with the build results of C code, mostly empty.
• _build\dll\libIoFoo.dll and libIoFoo.dll.a: Compiled C code, its dynamic library and GCC import library file.
• _build\headers: Empty for bundled addons, but should contain the C headers of exported functions, i.e. the Io*.h files from source.
• bin: Starter scripts for addons which are command line tools. e.g. Kano.
• io\*.io: Main Io source code, i.e. new prototypes. There is at least a Foo.io.
• samples\*.io: Optional sample code.
• source\*.c|h: C source code, at least IoFooInit.c. The empty folder must exist.
• tests\correctness\run.io and one or more *Test.io: Unit tests, the run.io runs all tests in the current folder.
• tests\performance\*.io: Optional performance tests, sometimes with run.io.
• depends file: a list of other prototypes (or addons) this addon depends on.
• protos file: a list of all prototypes this addon provides.

Addons are activated in Io by using the prototype with the name of the addon. In an Io REPL or script, Foo will find the addon and import it making all its prototypes available. This works for all provided prototypes given the manifest (i.e. the protos file) is correct.

Io Package Management: Eerie
While working with the different types of addons, see this and the next article, I seem to have reverse engineered much of Io's build and package infrastructure ;-). Knowing about it upfront would have helped. Eerie is the package manager for Io. Unfortunately its installation failed on my system - and it seemed way too complicated for me because it created multiple environments, much like Python's virtualenv. Its documentation is "coming soon", and not helping. Some addons on GitHub are Eerie packages and miss necessary files like protos. While sometimes these files are still there - an oversight as it seems, Eerie packages contain other files:

• package.json is the Eerie manifest. It contains the description of the package and its dependencies. This file is useful.
• eerie.json is some kind of manifest, used in the Eerie Package Storage Database. Only two addons have it and the database is empty.
• build.io will "rewrite AddonBuilder to represent a receipt for your package." Most addons have this and it contains information about needed native libraries and transitive dependencies.

It seems that Eerie's PackageInstaller extractDataFromPackageJson() method creates the protos, depends and build.io files from package.json. I will have to do this by hand. Unfortunately some package.json are incomplete and some miss important information.

Now I explain the installation of several addons with different needs and increasing difficulty using concrete examples:

Addons without Any Native Code: Kano
There are some addons without any C source, e.g. CGI, Continued​Fraction, Rational and integrations like Bitly, Facebook, etc. All of these are included in the distribution. When looking for addons, I checked all addon repositories for ones without native code and found Docio, Eerie and Kano. I guess these were left out because they are part of Eerie. Let's look at Kano. Kano is a simple Make/​Rake inspired tool for Io. It uses a Kanofile to declare tasks. Let's get it running:

> cd "%IO_HOME%\lib\io\addons"
> git clone https://github.com/IoLanguage/kano
Cloning into 'kano'...
> io -e "Kano println"
Exception: Object does not respond to 'kano'

This is broken. io -e "<expression>" runs Io and passes the command, much like Ruby does. Passing the prototype name asks Io to load it and it is not found. By Io convention prototypes start with an uppercase letter while instances (and fields and variables) start with a lowercase one. Somewhere (in the source of Importer maybe) I read that the names must match for importing. Also this is one of only three repositories which lowercase name, i.e. docio, eerie and kano. Tip: Watch out for addon repository names matching the primary prototype.

> ren kano Kano
> io -e "Kano println"
Exception: unable to read file '...\lib\io\addons\Kano\depends'

Ah progress. It is missing the depends and protos files I mentioned earlier. It has a package.json instead:

{
  "version": 0.1,
  "author": "Josip Lisec",
  "description": "Rake-like utility for Io.",
  "readme": "README.textile",
  "category": "Utility",
  "dependencies": { },
  "protos": ["Kano", "Namespace", "Namespaces", "ObjectDescriber"]
}

No dependencies and four prototypes.

> touch Kano\depends
> echo Kano Namespace Namespaces ObjectDescriber > Kano\protos
> io -e "Kano println"
Exception: Unable to open directory ...\lib\io\addons\Kano\source

Progress again but why does Io look for C sources? In some addons, I see an empty source folder with a .gitisdumb file in it, to keep the folder in version control. Maybe it is needed.

> mkdir Kano\source
> io -e "Kano supportedFiles println"
list(make.io, Kanofile, kanofile, Kanofile.io, kanofile.io)

Nice, no error. Kano works. To be fully useable, there are a few more things to fix:

1. Kano is a tool, it comes with bin/kano starter script. Each starter script needs a Windows version, i.e. a bin/kano.bat which calls the original starter script,

   io "%IO_HOME%\lib\io\addons\Kano\bin\kano" %*

   And both scripts need to be in the PATH. The easiest way is to copy kano.bat to %IO_HOME%\bin, which contains io.exe.
   
   
2. Kano offers kano [-T|V|help|ns] [namespace:]taskName arg1 arg2... as help. -T lists all tasks defined in the local Kanofile and -V shows its version. At least it tries to do that. It calls Eerie to get its version. Now Eerie depends on Kano and Kano depends on Eerie, and I do not have Eerie, this is all bad. If you want everything to work, you can replace line 44 in Kano's Namespaces.io:

    V := option(
      """Prints Kano version."""
   -  version := Eerie Env named("_base") packageNamed("Kano") \
                       config at("meta") at("version")
   +  version := File thisSourceFile parentDirectory parentDirectory \
                      at("package.json") contents parseJson at("version")
      ("Kano v" .. version) println)

3. Much later I noticed that Kano's package.json does not have a name field. This is needed for Docio to generate the documentation. Sigh. (Here is my patched Kano.)

Follow along in part two.

Shut up and write a test!

Due to the hype around ChatGPT I wanted to create this text with it. I failed. I tried several versions of different prompts, and also asked the Bing variant, but it failed to create any text I liked. It could not simulate my personal style. I should have to train a model with all my blog posts ... It seemed easier to use with the good old method of writing my texts myself.

Should I Write a Test?
Around ten years ago I stumbled upon a diagram if and when to write a test. This was a repeating question in trainings, when should you write tests? Before or after writing the code? Only when you have time? Only when you feel like it? I loved the diagram because there was only one end state. Whatever the reason or excuse would be, just shut up and write a test! It does not matter if you are tired, or if it is a little change, it should be tested. Such a rule is helpful when you want to adopt Test Driven Development, as there are plenty of excuses why to skip it.


The original diagram was created by Gemma R Cameron, a software engineer and event organiser from Manchester. She was inspired by a similar diagram about skating. Recently, when I was asked again when to test, I wanted to share the diagram and could not find its page. I thought it lost and planned to repost the diagram here. Eventually I discovered Gemma's page again, it had moved from a .me to a WordPress domain. I am glad that the original source is still available. You can find it here.

Java Version
The original version of the diagram was for Ruby, indicated by the use of Automock. What was bad with Automock? It provided some support for automated mocking of Rails application and is dead since 2016. Here I present a slightly modified version for Java, replacing Automock with PowerMock. And I do know what is wrong with PowerMock. It is a powerful mocking tool which uses a custom class loader and bytecode manipulation to enable mocking of static methods, constructors, final classes and methods, private methods and more. It is useful to bring legacy code under test, before seams can be introduced. The bytecode manipulation makes it slow compared to regular mocking. If used often, the whole test suite will run much longer than needed. Further, if code is hard to test, I change its design to make it testable by applying SOLID principles or by creating seams as outlined by Michael Feathers in his WELC book. Using PowerMock for tests does not improve the design. (I am not inherently against PowerMock, it is extremely powerful and as such dangerous. I needed something to replace Automock.)

Download and print the Java version of the Shut up and Write a Test Diagram. The fact that there are versions of this diagram for both Ruby and Java is a testament to the universality of its principle: Shut up and Write a Test! The Code Cop approves, and I am sure your future self will too.

Learning yet another Programming Language

In 2000 Andrew Hunt and David Thomas wrote their influential book The Pragmatic Programmer, which is listed as second single most influential book every programmer should read. (I listed it in my book recommendations both 2012 and 2006.) Chapter one, tip eight Invest Regularly in Your Knowledge Portfolio says: Learn at least one new language every year. Different languages solve the same problems in different ways. By learning several different approaches, you can help broaden your thinking and avoid getting stuck in a rut.

I started out as Java developer. Since than I have studied XSLT, Ruby, Scala, Forth, JavaScript, Scheme, Python, TypeScript, Go, C# and C. Out of these I even ran trainings for developers teaching them Python and TypeScript. In addition I had a glimpse of Visual Basic, Dart, Clojure, R, PHP, NATURAL, PowerShell, Kotlin and have relearned Assembly. I still need to learn Smalltalk, F#, maybe Haskell, J and of course Prolog.

I like programming languages and my learning approach is driven by curiosity and fun. Today I will describe my "standard", step by step way to get into a new language. I will assume you are an experienced developer, able to code in Java or C# who understands basic programming principles. I believe this approach is not suitable for programming newbies. I never use all these steps and sometimes change their order - like starting with the last one. Feel free to reorder or skip any steps not adding knowledge or fun.

1) Get an overview of core language features.
I start reading Wikipedia about the programming language I want to dive into. I am looking for the core features, used concepts and paradigms in the language. Code examples of these features provide a first idea of the language's syntax. The idea is not to know everything, just to be able to write some code. Writing code is the fun part, not reading. The more languages you know the easier and faster this step is. When learning Go one hour on Wikipedia during commute was enough. On the other hand for TypeScript I spent several hours reading the language reference (Handbook). For C# I skipped this step as I had seen most of the language features while facilitating Mob Programming sessions. You are done with this step when you have some idea what the language can do.

2) Figure out the usual setup and get it working.
Most programming languages come with their own ecosystem of runtimes, documentation, dependency and packaging mechanisms, testing frameworks, editors and other tools. When starting with a new language I try to use its typical tooling. Or at least I look for a decent IDE plugin to keep some level of comfort and productivity. Often this is painful and full of compromise. E.g. when working with Scheme I should have used Emacs but started with a basic editor and used VS Code in the end. For Go I needed 2 to 3 hours to set up the command line tools and VS Code integration. For C it took me 2 hours to compile and run a sample test alone - due to incompatible architecture binaries, sigh. I am still staying away from make tools due to the additional complexity. Eventually I will have to work through them if I want to use high level IDEs like Eclipse or CLion. If your interest in the language is purely educational, a simple editor might be enough to get started, like I used for Forth. You are done when you are able to edit, compile, test and run a Hello World application, e.g. in Windows Assembly.

3) Port small pieces of code (from a similar language if possible).
This helped me when learning C. I knew its basic features and had figured out how to compile and run a single file. And then I ported some small (refactoring) code katas. Porting code katas was easier than coding them because the solution and its code were already there and all I needed to deal with was syntax. If there is a similar language to start from, e.g. Java to C#, Java to PHP, C++ to C, even Java to C, then some of the syntax is proper right from the start. There was a lot of Google and StackOverflow involved and I managed to convert a small kata in around one hour. In the end I had ported several code katas to C. Emily Bache keeps inventing fun and interesting refactoring katas, which always need ports to other languages. For example I have ported her Parrot-Refactoring-Kata to TypeScript, Go and recently C. I have also contributed a Scheme version, but that was after I had learnt the language. You are done when you have contributed at least two ports of small refactoring katas.

4) Work through koans of the language.
Koans are an effective way to learn new programming languages. Programming language koans are a progressive sequence of little exercises, starting with basic things and building on each other to move to more advanced topics. The goal is to learn the language and core libraries. Usually the exercises contain failing test cases, where tiny pieces of code have to be filled in to make them pass. (I have used this idea to teach unit testing in Java and PHP as well as Python and C#. Porting the koans from JUnit to xUnit also helped me to get into C# - see the paragraph on porting small pieces of code.) Koans are awesome and I use them regularly. I worked through some Python koans, two third of Kotlin koans and all C Koans. The koans vary in size. While Kotlin contained 6 lessons, Python had 39 and JavaScript even 78. Working all these 78 exercises took me more than three full work days in total. You are done when you have completed (almost) all of the koans for the new language.

Ninety-Nine Problems
In case there are no koans for your language, look for "Ninety-Nine Problems". The idea is based on Werner Hett's P-99: Ninety-Nine Prolog Problems. These are little problems with different levels of difficulty. Sample solutions are available in Java, Scala, Haskell, Kotlin, F#, OCaml and probably others.

5) Watch some recorded talks and online presentations.
This is the most obvious step. I like to watch recorded presentations, during my commute. Sometimes some extra (passive) information helps me understand a language, or even this specific weird feature. Depending on time and interest this can be a few talks, or whole months of commute.

6) TDD some code katas from scratch.
Now is the time to write some code from scratch. I recommend starting with simple exercises. There is no point in getting frustrated right at the start. Manage the difficulty of your exercises: Use simple katas like FizzBuzz, Prime Factors, Roman Numerals or Word Wrap to get started. For example I used FizzBuzz to practice some XSLT and Prime Factors when revisiting old languages I used to know many years ago - like BASIC. I reuse these katas - and know their solutions - I just want to try them in a new language. Later I move to more complicated assignments like Bowling, Minesweeper (when I revisited Assembly after 20 years), Bank OCR (when I studied Scheme) or Conway's Game of Life. You are done when you have implemented a few code katas from scratch including tests.

7) Write more code, e.g. tackle a larger code base.
There is no other way of getting deeper into a programming language than using it. This calls for a larger side project. It can be a complex code kata, a little video game like Pong or Tron (Nibbles) - including graphics of course - or whatever comes to your mind. As I am fond of Scheme, I have build several Scheme interpreters in new languages. You could even implement your own unit testing framework, which is also recommended by Kent Beck as an exercise to get into a new language. (And I did that for Pascal, Assembly and Scheme.) Depending on the time invested into the previous steps, a side project includes more or less trial and error. If there is too much hassle, I stop and go back to previous steps to learn more about the basics. There is no point in being stuck. For example when studying Go, I went for the side project after 3 hours of researching the language, which was too early. So I spent some more time on theory and then continued working on my idea. My usual learning side projects take around 15 to 20 hours - or that is the time when I lose interest. You are done when you have worked on a larger code base for at least 15 to 20 hours.

8) At last get *all* the details.
Now that I am familiar with the basics of the language and its ecosystem, it is time to dive deeper. After some month of experimenting and hands-on practice - steps 2, 3, 4, 6 and 7 all involve writing code - I am drawn back to theory. I like to balance my learning between theory, experiments and practice. To conclude learning a new language I might study a classic book about that language. It should cover the language and its features completely and I read it from cover to cover. At that time I am already familiar with many parts of the language, reading progress is fast. I am interested in the all the details, the bits and pieces I did not encounter during my experiments. Language specifications are usually dry and perfectly suited for this step as well as classic titles like the "Pickaxe book", K&R and SICP (although SICP is so much more than a Scheme book...) You are done when you read a classic book on the new language from cover to cover.

Conclusion
I like programming languages and learning new ones is adventurous and fun. I try to learn a new language every year. Not all learning goes deep. Not all languages stick. Unless you are working on real projects in all of these languages at the same time, it is natural to forget some details. And that is perfectly fine. It is all about incorporating new paradigms and widening your perspective. So keep learning!

TDD Classic State Based UI

Most modern user interface technologies are state based. Classic frameworks like Swing, JavaFX, Eclipse RCP, Windows Forms, Vaadin and many more consist of heavy weight UI widgets that manage all their state and interact with the application using event handlers. Such frameworks are harder to test than plain code. I have not found much information on how to do that, one resource that got me started is chapter eight from Lasse Koskela's Test Driven. I am using many ideas from that book in here.

Model View Presenter (MVP) Pattern
Because these UI technologies are heavy weight, hard to test and often slow to run, one needs to decouple from them as much as possible. The goal is to minimize all UI dependencies. As few classes as possible should be "contaminated" with UI stuff, and the code that needs to do UI should contain as little logic as possible. Common ways to separate concerns between the interface and the underlying logic are MVC, MVP, MVVM or similar architectural patterns. I am using the MVP pattern. Its goal is - obviously - to facilitate automated unit testing and improve the separation of concerns in presentation logic. That means MVP is used to have a very thin, dumb UI, which is called the view. The model contains the UI data or UI model which might contain information about enabled fields, active buttons and so on. The presenter is a mediator and wires the model and the view.

Using Java and Swing
On my research on how to test drive user interfaces I started with Java and Swing. The Wikipedia page lists several UI frameworks which use the MVP patterns themselves, Swing being one of them. That means that Swing uses MVP internally. Larger components like JComboBox or JTable have their own model classes ComboBoxModel and TableModel. This is useful because the models do not depend on the user interface part of Swing and can be TDD'ed in the usual way. It does not mean that Swing can be test driven easily. As example I will run my Login Form Kata. I am going to develop the basic login screen step by step using TDD.

First Test: LoginModel should contain user lookup and password
I start with the model. The model contains the state of the user interface. There is no problem using TDD for that. My first test is

class LoginModelTest {

  LoginModel model = new LoginModel();

  @Test
  void shouldContainUserLookupAndPassword() {
    model.setLookup("user@server.com");
    model.setPassword("secret123");

    assertEquals("user@server.com", model.getLookup());
    assertEquals("secret123", model.getPassword());
  }
}

After seeing it red, I add

public class LoginModel {

  private String lookup;
  private String password;

  // getters and setters for lookup and password
}

Green. Usually I am not testing for getters and setters, but today I have to start somewhere. Depending on the used version of MVP, the model could update the view itself. I choose to keep the model very simple, following the Passive View aka Humble Dialog Box variant of MVP. This seems to be the usual way. In this example of login form, there is no state that goes back from the model into the view yet, e.g. if the login button should be disabled, so it does not make any difference.

Test: LoginPresenter should pass lookup and password to its model
Next is line is the LoginPresenter. The presenter is handling all user input. It is important, that the presenter has no dependency on Swing, too, so there is no problem TDDing that. The presenter will be notified of user input into the lookup or password field and will store the values in the model.

class LoginPresenterTest {

  LoginModel model = new LoginModel();
  LoginPresenter presenter = new LoginPresenter(model);

  @Test
  void shouldPassLookupAndPasswordToModel() {
    presenter.lookupChanged("user");
    presenter.passwordChanged("pass");

    assertEquals("user", model.getLookup());
    assertEquals("pass", model.getPassword());
  }
}

public class LoginPresenter {

  private final LoginModel model;

  public LoginPresenter(LoginModel model) {
    this.model = model;
  }

  public void lookupChanged(String newLookup) {
    model.setLookup(newLookup);
  }

  public void passwordChanged(String newPassword) {
    model.setPassword(newPassword);
  }
}

Empty LoginView
Finally I create the LoginView interface. The view wraps the UI technology completely. It starts as an empty interface. The methods to come will be driven by the needs of the presenter.

public interface LoginView { }

Test: LoginPresenter should close the view on successful login
It is time to go for some real logic. When the login button is clicked, the authentication back end will be called and if the call was successful, the view should be closed. The test uses Mockito to mock the view and verify it has been called.

import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;

class LoginPresenterTest {

  LoginModel model = new LoginModel();
  LoginView view = mock(LoginView.class);
  AuthenticationService auth = mock(AuthenticationService.class);

  LoginPresenter presenter = new LoginPresenter(model, view, auth);

  @Test
  void shouldCloseViewOnSuccessLogin() {
    model.setLookup("user");
    model.setPassword("secret");
    when(auth.authenticate("user", "secret")).
      thenAnswer(invocation -> {
        return new AuthenticationResult(true, null);
      });

    presenter.loginButtonClicked();

    verify(view).close();
  }
}

This adds a close() method to the LoginView. Because I own the view, I can use domain names and the actual UI technology is not visible from the outside.

Test: LoginPresenter should display an error on failed login
Another test drives the display of an error message if authentication fails. This creates a showError(String message) method in the view.

  @Test
  void shouldDisplayErrorOnFailedLogin() {
    model.setLookup("user2");
    model.setPassword("secret2");
    when(auth.authenticate("user2", "secret2")).
      thenAnswer(invocation -> {
        return new AuthenticationResult(false, "Login failed.");
      });

    presenter.loginButtonClicked();

    verify(view).showError("Login failed.");
  }

And so the loginButtonClicked method is complete.

  public LoginPresenter(LoginModel model, LoginView view,
                      AuthenticationService authenticationService) {
    this.model = model;
    this.view = view;
    this.authenticationService = authenticationService;
  }

  @Override
  public void loginButtonClicked() {
    AuthenticationResult result =
      authenticationService.authenticate(model.getLookup(), model.getPassword());
    if (result.success) {
      view.close();
    } else {
      view.showError(result.message);
    }
  }

Test: LoginPresenter should call Authentication service asynchronously
As stated in the requirements, all calls to the back end need to be asynchronous. Synchronous code would block the Swing event thread and render the UI unresponsive. So I change the tests to force the calls to be asynchronous. I need two java.​util.​concurrent.​CountDownLatchs and have to wait on them to start and finish asynchronous processing.

Test Bed
The last piece is the view - the implementation of the view to be precise. To test the real view it has to be started and sent some events. This is what I wanted to avoid. It is slow and brittle. By following the MVP pattern described above, the view will have as little code as possible. There will only be a few tests executing it. Sometimes it is necessary to call certain methods on UI classes for the UI to work at all, e.g. show. It is not pretty. Best would be to use a test bed or harness to run these UI components in an isolated way, e.g. to have a dedicated window for the component under test and to create the application in a way that allows to exercise UI features independently.

Lasse Koskela mentions Abbot to test stand alone AWT or Swing components. It provides helper code for finding and interacting with UI elements. See how Abbot works in one of its tutorials. It is pretty old and its test support is for JUnit 3, but it gets the job done. It displays a frame while running the tests and there are flickering tests from time to time. This is not a problem of Abbot itself, but is the nature of full UI tests. As I said, these are brittle.

Test: SwingLoginView has a login button with text
A basic start is to assert the existence of UI elements. Abbot offers several ways to find elements, using its getFinder().​find(...) method. An easy way to locate elements is to use ids or names, e.g. with loginButton.​setName("LoginButton").

import javax.swing.JButton;
import javax.swing.JPanel;

import abbot.finder.ComponentSearchException;
import abbot.finder.matchers.NameMatcher;
import junit.extensions.abbot.ComponentTestFixture;

public class SwingLoginViewTest extends ComponentTestFixture {

  LoginView view = new SwingLoginPanel();

  public SwingLoginViewTest(String name) {
    super(name);
  }

  public void testHasLoginButtonWithText() throws ComponentSearchException {
    showFrame((JPanel) view); // Abbot shows the view
    JButton loginButton = findLoginButton();

    assertEquals("Log in", loginButton.getText());
  }

  private JButton findLoginButton() throws ComponentSearchException {
    return (JButton) getFinder().find(new NameMatcher("LoginButton"));
  }
}

This tests forces me to create the initial login panel.

import javax.swing.JButton;
import javax.swing.JPanel;

public class SwingLoginPanel extends JPanel implements LoginView {

  private final JButton loginButton = new JButton("Log in");

  public SwingLoginPanel() {
    createLoginButton();
  }

  private void createLoginButton() {
    loginButton.setName("LoginButton");
    add(loginButton);
  }

  @Override
  public void close() {
  }

  // other empty LoginView methods

}

Observer Pattern
In MVP, the view delegates user input somewhere else and does nothing but rendering. We using the Observer Pattern to get notifications from the UI. The view takes the role of subject and the presenter is observing it. Observer Pattern subjects need to manage the list of observers - in Java usually named listeners - and allow code to register and sometimes deregister them.

Test: SwingLoginView should send button click to presenter

import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.verify;
import abbot.tester.JButtonTester;

...

  public void testSendButtonClickToPresenter() throws ComponentSearchException {
    LoginListener listener = mock(LoginListener.class);
    view.registerLoginListener(listener);

    showFrame((JPanel) view);
    JButton loginButton = findLoginButton();
    JButtonTester tester = new JButtonTester(); // from Abbot
    tester.actionClick(loginButton);

    verify(listener).loginButtonClicked();
  }

As I said, the presenter will be the actual observer, hiding behind the LoginListener interface.

public interface LoginListener {

  void loginButtonClicked();

}

Its implementation is straight forward.

  @Override
  public void registerLoginListener(LoginListener listener) {
    loginButton.addActionListener(ae -> listener.loginButtonClicked());
  }

Test: SwingLoginView has input fields for lookup and password
Like for the login button, I assert the existence of lookup and password fields and that they are wired to the listener. To verify that, the test code enters some text into the fields and checks that the listener mock has been called.

  public void testHasLookupField() throws ComponentSearchException {
    LoginListener listener = mock(LoginListener.class);
    view.registerLoginListener(listener);

    showFrame((JPanel) view);
    JTextField lookupField = findLookupField();

    assertEquals(20, lookupField.getColumns());

    // verify that it is wired as well
    JTextFieldTester tester = new JTextFieldTester(); // from Abbot
    tester.actionEnterText(lookupField, "user");

    verify(listener).lookupChanged("user");
  }

  private JTextField findLookupField() throws ComponentSearchException {
    return (JTextField) getFinder().find(new NameMatcher("LookupField"));
  }

Yes, this is not a good unit test as it tests two things - visible by its two checks, assert and verify. I should split that into two independent tests. Unfortunately these tests are slow, so I want to minimize their number. Together with a similar test for the JPasswordField passwordField the view is taking shape.

  private final JTextField lookupField = new JTextField(20);
  private final JPasswordField passwordField = new JPasswordField(20);

  private void createLookupField() {
    lookupField.setName("LookupField");
    add(lookupField);
  }

  private void createPasswordField() {
    passwordField.setName("PasswordField");
    add(passwordField);
  }

  @Override
  public void registerLoginListener(LoginListener listener) {
    lookupField.getDocument().
      addDocumentListener(new AllDocumentListener() {
        @Override
        protected void fire() {
          listener.lookupChanged(lookupField.getText());
        }
      });

    passwordField.getDocument().
      addDocumentListener(new AllDocumentListener() {
        @Override
        protected void fire() {
          listener.passwordChanged(new String(passwordField.getPassword()));
        }
      });

    loginButton.addActionListener(ae -> listener.loginButtonClicked());
  }

To be notified of each input character, I use Swing's DocumentListener. (AllDocumentListener is my own base class to handle the many methods of it.) This gives the final listener for the observer:

public interface LoginListener {

  void lookupChanged(String lookup);
  void passwordChanged(String password);
  void loginButtonClicked();

}

Test: SwingLoginView should display errors
This is the final test for the view.

  public void testErrorDisplay() throws ComponentSearchException, InterruptedException {
    showFrame((JPanel) view);
    JLabel errorField = findErrorField();

    assertEquals("", errorField.getText());

    view.showError("Alert!");
    Thread.sleep(10); // wait for asynchronous update

    assertEquals("Alert!", errorField.getText());
    assertEquals(Color.RED, errorField.getForeground());
  }

  private JLabel findErrorField() throws ComponentSearchException {
    return (JLabel) getFinder().find(new NameMatcher("ErrorField"));
  }

And the final SwingLoginView:

import java.awt.Color;

import javax.swing.JLabel;
import javax.swing.SwingUtilities;

...

  private static final Color ERROR_COLOR = new Color(255, 0, 0);
  private final JLabel errorField = new JLabel();

  private void createErrorField() {
    errorField.setName("ErrorField");
    errorField.setForeground(ERROR_COLOR);
    add(errorField);
  }

  @Override
  public void showError(String message) {
    SwingUtilities.invokeLater(() -> errorField.setText(message));
  }

SwingUtilities.invokeLater() is needed because showError is called asynchronously. While there is agreement in the TDD community to not assert on layout or styling, I am asserting that the foreground of the error field changed to an colour indicating the error, because it is important to indicate errors also in colour.

Test: LoginPresenter should register itself to the view
To complete the Observer Pattern, the presenter needs to register itself to the view. I started with verify(view).​registerLoginListener(​Mockito.​any(​LoginListener.class)); but that is not what I want. I really want that the methods on the listener trigger the required functionality in the back end or model.

import org.mockito.ArgumentCaptor;

...

  @Test
  void shouldRegisterItselfToView() throws InterruptedException {
    when(auth.authenticate(any(String.class), any(String.class))).
      thenReturn(new AuthenticationResult(true, null));

    ArgumentCaptor<LoginListener> argument = ArgumentCaptor.forClass(LoginListener.class);
    verify(view).registerLoginListener(argument.capture());
    LoginListener listener = argument.getValue();

    listener.lookupChanged("user");
    assertEquals("user", model.getLookup());

    listener.passwordChanged("pass");
    assertEquals("pass", model.getPassword());

    listener.loginButtonClicked();
    Thread.sleep(10);
    verify(auth).authenticate("user", "pass");
  }

When the presenter registers itself to the view, I capture the listener using Mockito's ArgumentCaptor. Then I call the listener and check the wanted behaviour. The sleep time is required because authentication is run asynchronously. Finally the presenter is complete.

public class LoginPresenter implements LoginListener {

  // ...

  public LoginPresenter(LoginModel model, LoginView view,
                        AuthenticationService authenticationService) {
    this.model = model;
    this.view = view;
    this.authenticationService = authenticationService;

    view.registerLoginListener(this);
  }

  // ...
}

and the final view (interface) is

public interface LoginView {

  void close();
  void showError(String message);
  void registerLoginListener(LoginListener listener);

}

The actual commits are here.

Conclusion
This was my first try and it worked well. I conclude that it is possible to TDD UIs at least using Java and Swing. I have working examples and the testing tools are fair. The final code has the usual TDD benefits - more separation between concerns (i.e. logic and UI technology) and a better design using domain methods. I tested the UI elements briefly. I did not go into test driving styling, colours or positions. In Swing components have all these properties accessible, e.g. visibility, colours, positions and more. So I could have asserted them. That is nice because I could go as far as I wanted to. I am undecided if I should test colours and other styling related things. As I said, there seems to be consensus to not assert on layout or styling and I did not. In general it is still unclear (for me) how much automated testing is needed for an UI in such situations. While I did not write tests for everything, I already have a feeling that there are "lots of test but no application".

Coderetreat Facilitation Podcast

tl;dr: I am podcasting about Coderetreat facilitation.

I regularly facilitate Coderetreats - as part of my work as Code Cop and for the local Software Crafters community. To support people organising events in other cities, I run Coderetreat facilitation trainings each year, especially in the months before the Global Day (of Coderetreat). Based on the feedback I get, e.g. Thanks for the training today! Very helpful! I see that these trainings are helpful for people who want to run Coderetreats. Often people ask the same questions, so I decided to record the most common questions. I will publish the recordings as I create them, so it is a podcast. (A podcast is an episodic series of digital audio files that a user can download in order to listen.)

Get it here: coderetreat-facilitation.code-cop.org

About
In the podcast I am answering questions about Coderetreat hosting, facilitation and participation. It will help you run Coderetreats, Coding Dojos, hands on meetups and even classic training. Each episode covers two to three questions and takes up to 10 minutes. From time to time I will invite guest facilitators to discuss with me. In the first few episodes I will also cover basic questions about the Coderetreat itself, e.g. What is a Coderetreat?

Frequency
As each episode is short, I plan to release at least two each month. While I am too late for #GDCR19, there should be around 30 questions and answers ready for people who want to help organising the next #GDCR20. That would be like three hours facilitation training. That would be great. Let's see how far I get.

Questions
If you have any questions regarding your Coderetreat, Coding Dojo or hands-on workshop, please send me an email or leave a comment. I will answer the question in one of the next episodes.

Get it here: coderetreat-facilitation.code-cop.org

Scheme Programming Language

With the beginning of this year I "fell" into Scheme. I blame SoCraTes BE 2015 for that. (SoCraTes is a group of international unconferences focusing on Software Craft and Testing. They are about sustainable creation of useful software in a responsible way and usually run as self-organised Open Space.) During that SoCraTes I happened to enter a session where a small group worked on some Scheme code. The Scheme programming language is one of the two main dialects of Lisp and I recognise Lisp when I see it. The session was so much fun that it continued for the whole of the second day. We used available slots in the schedule of the open space, and in cases there were none, we continued coding Scheme in the hallway, the kitchen and other places. (Now such things only happen on SoCraTes and I encourage you to attend one if possible.)

This SoCraTes experience broke the ice with Scheme, i.e. it gave me enough exposure to a new language and enabled me to continue on my own. For the next two years I kept playing with Scheme, experimenting with code katas, e.g. the Bank-OCR Kata, porting well known exercises like Parrot, Gilded Rose and Game Of Life and I even solved some Project Euler problems using Scheme.

SICP
Earlier this year I started reading SICP. SICP stands for Structure and Interpretation of Computer Programs, a computer science text originally used in introductory courses taught at the MIT in the eighties. It had been on my reading list since 2010, when Uncle Bob recommended it in one of his Clean Coder videos. SICP had been recommended again and again and I was very happy to find the time and energy to read it.

SICP is one of the greatest books - if not the greatest book - I have ever read. It is fast paced and written in the style of all scientific books - too dense and hard to follow. Reading it is hard work. I used to like such books when I was studying applied mathematics many years ago - actually I was eating such books for breakfast ;-) The high density gives me new information fast. As I like to read books from cover to cover, I do not mind that the information is not accessible directly. Yes it is challenging to read. And it is very insightful. I appreciate most the skill of the authors to define abstractions. I am stunned again and again how they name their composed methods.

The Scheme Programming Language
SICP is strongly related to Scheme as it is one of the bibles of the Lisp/Scheme world. SICP uses Scheme for its code samples and exercises as Gerald Jay Sussman, one author of SICP, is also one of the creators of Scheme. Scheme is a version of Lisp and as such a divine language per se. It is a great language and unlike Lisp it follows a minimalist design philosophy. I really like its simplicity: There are no reserved words, not much syntax, just atoms, pairs and lists.

(define atom?
  (lambda (x)
    (and (not (pair? x))
         (not (null? x)))))

Atoms are numbers, strings, booleans, characters and symbols. Symbols i.e. names usually represent other atoms or functions. Maybe you do not like all the extra parenthesis, but it is compact and uniform. Because of the uniformity of the S-expression, it is easy to create parsers, embedded languages and even full featured Schemes, e.g. in Python, Haskell or any language. (To be fair, the last link contains implementations of Lisp not Scheme in 73 languages.)

History
As I said before Scheme is based on Lisp. The Lisp programming language was created 60 years ago by John McCarthy. And Lisp is very special, it seems to transcend the utilitarian criteria used to judge other languages. In 1975 Scheme was created by Gerald Jay Sussman and Guy Lewis Steele. Watch this great talk by Guy Steele about issues associated with designing a programming language.

Standards
One thing which confused me a lot when I started playing with Scheme were its versions. The Scheme language is standardised by IEEE and the de facto standard is called the Revised n Report on the Algorithmic Language Scheme (RnRS). The most widely implemented standard is R5RS from 1998. I use that mainly because the first Scheme implementation I downloaded was an R5RS compliant one. R5RS files use the .scm filename extension. The newer R6RS standard, ratified in 2007, is controversial because it departs from the minimalist philosophy. It introduces modularity which is breaking everything. Being used to the strong compatibility of Java or at least the major compatibility of Python I did not expect versions to be incompatible at all. R6RS files use the .ss, .sls and .sps filename extensions. The current standard is R7RS from 2013, a smaller version, defining a subset of the large R6RS version retaining the minimalism or earlier versions.

(Almost) Complete List of Special Forms and Functions
I did not read any tutorials or books on the Scheme language - I was exploring the language on my own. Still I wanted to know which library functions I could use. Similarly I used to list all classes available in each Java release. I searched for an exhaustive list of Scheme functions. There are many Schemes and I did not want to depend on any implementation specific extensions. In the end I decided to scrape the standards.

Scheme makes a difference between forms and functions. Function calls evaluate all arguments before control is passed to the function body. If some expressions should not be evaluated - as in if or switch (which is cond in Scheme) - a special form is needed. Special forms evaluate their arguments lazily. For more information see Why is cond a special form in Scheme.

R5RS Forms and Functions
I scraped the list of special forms and built-in functions from the R5RS HTML documents. The list is incomplete since I had to rely on formatting of the document. It misses define and the abbreviations like ' and @, but looks pretty good (to me) otherwise. Browsing the 193 forms and functions gives an idea of built in data types, i.e.

boolean
char
complex (number)
exact (number)
inexact (number)
list
number
pair
procedure
rational (number)
string
symbol
vector

as well as possible conversions between them available in any R5RS compliant Scheme.

char->integer
exact->inexact
inexact->exact
integer->char
list->string
list->vector
number->string
string->list
string->number
string->symbol
symbol->string
vector->list

R6RS Forms and Functions
As mentioned earlier, R6RS is much larger than R5RS. It defines 630 forms and functions, most of them in the (new) libraries. The standard separates built-in forms and functions from the ones defined in libraries. (This is still very small, considering that the Java 8 core library contains 6000 public classes.) My list of forms and functions contains all, built-in and library alike. It looks complete, but I am not sure, I did not work with R6RS. From a quick glance R6RS adds bitwise and floating point operations as well as different types of byte vectors, enum-set and hashtable. When looking at my list now, I see that I should have scraped the names of the library modules as well. (I added that to my task list for 2019 ;-)

Scheme Requests for Implementation
Next to the RnRS standards, there are the SRFIs, a collection of concrete proposals and reference implementations. People implementing Scheme chose to implement SRFIs or not. Most Schemes support some of then, see Arthur A. Gleckler's report of SRFI support by Scheme implementations in 2018. Some Schemes have package managers which allow to download and instal packages. Usually some of those are SRFI implementations. I guess I also need to scrape these.

Conclusion
Scheme (Lisp) is so much fun, especially when you do not have to deliver anything. Most people I meet got in touch with Lisp during university but never followed up. Some are even afraid of it. Since 2016 I run Scheme coding sessions at every unconference or SoCraTes event I attend. I invite people to mob with me and I do all the typing. We always have a good time, and - after some warm up with Scheme - people really like it. They all enjoy the opportunity to dive into Lisp again. Will you?

What is Ethical Coding and how to get involved

Three years ago I started thinking about work with meaning. Since then I had great discussions with peer Software Professionals about our social responsibility and ethics in software development. Today I am happy to present a guest post written by Kayleigh Alexandra, a content writer for Micro Startups, a site dedicated to spreading the word about startups. I like seeing a focus on ethical approaches in startups, i.e. Social entrepreneurship. Kayleigh Alexandra shares her ideas about ethical coding in this introductory article. For her latest startup news and inspiring stories, visit the Micro Startups blog or follow along on Twitter.

Technology provides amazing possibilities, but its power also makes it a threat. Almost everything we do relies on complex interactions of computer systems, with those systems running on code from countless distinct sources - you cannot trust one chunk of code to meet the same standard as another.

Unknown quality is not the only concern we can have about code, however. A function devised to achieve one thing can be extended and repurposed to do numerous other things - does not the developer bear some responsibility in how it is used? Those who share that sentiment argue for ethical coding, targeting a future of much broader social awareness. But what exactly does ethical coding involve, what are some relevant cases to consider, and how can you get involved in it? Let's take a look.

What does ethical coding involve?
Ethical coding involves acknowledging and acting upon the social responsibilities of a developer - factoring it alongside corporate responsibility - seeking to adhere to a set of meaningful values. It has becoming more prominent in recent years due to rising social awareness in general.

Because of the layer of abstraction between low-level source code and the consequences of the projects people work on, it is easy for everyone in a development company to absolve themselves of any culpability. Ethical coding requires the executive level to take ownership of the entire company output, committing to holding every last employee to a revised standard.

A developer with an ethical approach will think carefully about what her software will be used (or has been used) to achieve, and adjust course accordingly. They will also have strong thoughts about the general responsibilities of the software development industry, making them eager to advocate for higher standards and more transparency.

Since ethical stances vary wildly, though, there can be no precise definition of ethical coding - the best we can do is to use a broad ethical framework that loosely mirrors the societal standard for other fields such as medicine: minimising harm, being truthful, and making life better. See the Programming Ethical Guidelines by ACM and IEEE.

The rise of ethical software development
As noted, social awareness of the many issues that face the world continues to rise, driven by the 24/7 connectivity of the Internet and the move into adulthood of an ethically-conscious generation. For better or worse, social media users now act as moral guardians of sorts. Because technology in general plays a huge role in industry and the opaque nature of many coding projects is at odds with a desire for transparency, the tech world has attracted a lot of flak. This has led to various projects that have sought to demonstrate that technology can be a force for good, such as the following:

• The Hummingbird Clock. In 2016, as part of the Liverpool Biennial, this timepiece was installed as a commentary on government corruption and to serve as a tool for the public. Set up as three binoculars looking upon the Town Hall, it actually records the hum of the electrical grid and streams it online - since the government has long used that hum for surveillance, this was to make it available to anyone. As a result, anyone in legal difficulty can now use the hum to verify the time and date of a particular recording.
  
  
• The Empathy Deck. Social media can be brutal at times, deeply unpleasant and unempathetic, because being online and having some degree of anonymity drives people to embrace their darkest impulses. The Empathy Deck account was set up to respond to tweets with thought-provoking cards assembled from pieces of an artist's body of work, commenting on the perils of automation while providing some relevant musings and snippets of poetry to brighten someone's day.
  
  
• Cody Rocky. To many, the world of technology still feels quite dry and alienating, which can limit the types of people who choose to pursue it as a career. Cody Rocky is a programmable robot designed to appeal to children, lending a degree of accessibility to the field and making learning both productive and entertaining. It also crosses slightly into the nascent field of digital companionship.

In light of some high-profile fiascos making people much more concerned about the dangers of technology (such as the Cambridge Analytica scandal, or the death caused by a self-driven car), the pressure on developers to implement ethics policies has increased significantly. There is also the thorny issue of so-called hacktivism - is hacking done for a good cause any less objectionable? Who gets to decide where to draw the line?

Though industry-wide ethical frameworks have been suggested before, there is so much more impetus on companies to be proactive now. In the coming years, I think it is fairly likely that it will become standard for software developers - well, any companies in the digital industry - to lay out and commit to their ethical codes.

How to get involved in ethical coding
If you are a developer and you want to get involved in ethical coding, then you can. It is actually fairly simple, with the following two major steps:

• Code ethically. This is straightforward enough. Whenever you create applications, be keenly aware of any and all ethical issues that may arise as a result. For example credit everyone whose code you adapt, be efficient to avoid needlessly taxing a system, maintain professional integrity, and refrain from working on any project that you expect to be used irresponsibly or immorally. You cannot know every potential issue, but you can do your best.
  
  
• Contribute to ethical projects. These can be your own, or existing projects that need help. This is not about your code, but about the goals of the project: for instance, you could volunteer some time for a healthcare-related project, or help out a charity with its operations. It does not cost much to start your own business these days, so it is easy for someone with good intentions to launch a startup, but it is much harder to make it a contender. Your support could be the key to a business getting big enough to do some real good.

Beyond that, the specifics are entirely up to you. It is really a matter of getting involved with online communities, finding people who share your values, and slowly discovering how you can best use your time to engage in ethical coding.

Ethical coding is only going to grow as a concern while massively-influential fields such AI or the IoT get larger. If you develop software and are strongly ethical, give a lot of thought to how you can better reconcile your beliefs with your work. There is no reason why you cannot have professional and personal contentment at the same time.

IDE Support for Scripts

Last month I wrote about dealing with modified files during build in Jenkins. The solution uses Groovy scripts and the Jenkins Groovy plugin which allows execution of scripts in Jenkins' system context. These System Scripts have access to Jenkins' internal model like build status, changed files and so on. The model includes several classes which need to be navigated.

In many situations little additional scripts make our lives as developers easier: For example the Groovy System Script to customise Jenkins when building a NATURAL code base mentioned above, or a Groovy Script Transformation to perform repeated code changes in a large Java project using WalkMod, or a custom build during a Maven build, using any language supported by the Apache Bean Scripting Framework, e.g. Groovy, Ruby, Python or others. Now if the scripts are getting complicated IDE support would be nice.

Getting support for the language in the IDE
Depending on the language, most IDEs need installation of a plugin or extension. I will briefly describe the steps to set up Groovy support into Eclipse because it is more elaborate and the organisation I initially wrote this documentation used Eclipse. Getting plugins for IntelliJ IDEA is straight forward.

• First obtain the version of Eclipse you are using. Customised distributions like the Spring Tool Suite, IBM/Rational products or NaturalONE follow different version schemes than Eclipse. Navigate to the Help menu, item About, button Installation Details, tab Features, line Eclipse Platform to see the real version. For example, NaturalONE 8.3.2 is based on Eclipse 4.4.
• The Groovy plugin for Eclipse is GrEclipse. For each major version of Eclipse there is a matching version of GrEclipse. The section Releases lists the Update Site for each release. Find the suitable version. (If you use a newer Eclipse, maybe you have to use a snapshot version instead of a release version.) Copy the URL of the Update Site.
• The section How to Install explains in detail how to continue and install GrEclipse.

After that Eclipse can talk Groovy.

Enabling support for the language in the project
Next we configure the project for the additional language. In Eclipse this is often possible through the (right click) context menu of the project, menu item Configure. I usually do not bother and edit Eclipse's .project file directly to add the needed natures and build commands. Adding Groovy to a non-Java project needs adding to .project,

<projectDescription>
  <!-- existing configuration -->
  <buildSpec>
    <!-- existing configuration -->
    <buildCommand>
      <name>org.eclipse.jdt.core.javabuilder</name>
      <arguments>
      </arguments>
    </buildCommand>
  </buildSpec>
  <natures>
    <!-- existing configuration -->
    <nature>org.eclipse.jdt.groovy.core.groovyNature</nature>
    <nature>org.eclipse.jdt.core.javanature</nature>
  </natures>
</projectDescription>

and .classpath

<classpath>
  <!-- existing configuration -->
  <classpathentry exported="true" kind="con" path="GROOVY_SUPPORT"/>
  <classpathentry exported="true" kind="con" path="GROOVY_DSL_SUPPORT"/>
</classpath>

Similar changes are needed for Ruby (using org.eclipse.​dltk.core.​scriptbuilder and org.eclipse.​dltk.ruby.​core.nature) and Python (org.python.​pydev.PyDevBuilder and org.python.​pydev.pythonNature). If I am not sure, I create a new project for the target language and merge the .projects manually. Sometimes other files like .classpath, .buildpath or .pydevproject have to be copied, too.

Enabling support for libraries
Adding a nature to .project gives general support like syntax highlighting and refactoring for the respective language but there is no completion of library classes. Especially when using new libraries, like Jenkins Core, using code completion helps exploring the new API because you get a list of all possible methods to call. The trick is to add the needed dependency to the project in a way that it is not accessible from or packaged together with the production code.

An Example: Adding Jenkins System Script support to a NaturalONE project
To add dependencies to the NaturalONE project earlier, I converted the Java/Groovy project to Maven (by setting another nature and builder) and added the needed classes as fake dependencies to the pom.xml.

<dependencies>
  <!-- other dependencies -->
  <dependency>
    <groupId>org.jenkins-ci.main</groupId>
    <artifactId>jenkins-core</artifactId>
    <version>2.58</version>
    <scope>test</scope> <!-- (1) -->
  </dependency>
  <dependency>
    <groupId>org.jenkins-ci.plugins</groupId>
    <artifactId>subversion</artifactId>
    <version>2.7.2</version>
    <scope>test</scope>
  </dependency>
</dependencies>

Groovy has optional typing and after adding the types to the definitions and arguments, Eclipse will pick up the type and we get code completion on available fields and methods:

import hudson.model.Build
def Build build = Thread.currentThread()?.executable
build._ // <-- code completion, yeah!

This made developing and understanding the script to find modified files much easier. The Jenkins dependencies in the pom.xml are never used, as the System Script runs inside Jenkins where the classes are provided. The dependencies are declared to "trick" Eclipse into analysing them and providing type information. The scope of the dependencies is set to test (see line (1) in code snippet above) so the dependencies are not packaged and cannot be called from the production code. This zipped repository contains the NATURAL project together with all the Eclipse configuration files.

Another Example: Adding WalkMod Transformation support to a Maven project
Another situation where I wish for script support in Eclipse is when writing WalkMod Script Transformations. WalkMod is an open source Java tool that can be used to apply code conventions automatically. Read this tutorial by Raquel Pau to see how WalkMod works. WalkMod allows for Groovy scripts to define code transformations which manipulate the AST of Java classes. Navigating the AST is difficult in the beginning.

After adding the Groovy nature as shown in the previous example, the relevant dependencies to get code completion for the AST are

<dependencies>
  <dependency>
    <!-- API -->
    <groupId>org.walkmod</groupId>
    <artifactId>walkmod-core</artifactId>
    <version>3.0.4</version>
    <scope>test</scope>
  </dependency>
  <dependency>
    <!-- AST -->
    <groupId>org.walkmod</groupId>
    <artifactId>javalang</artifactId>
    <version>4.8.8</version>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>com.squareup</groupId>
    <artifactId>javapoet</artifactId>
    <version>1.10.0</version>
    <scope>test</scope>
  </dependency>
</dependencies>

In case of Walkmod transformations, the interesting class is (org.walkmod.​javalang.ast.​CompilationUnit) and it is already in scope as variable node. To add type information we need to alias it:

import org.walkmod.javalang.ast.CompilationUnit
def CompilationUnit cu = node
cu._ // <-- code completion on available fields and methods

Conclusion
Supporting scripts in Groovy, Ruby or Python make our lives easier. With some additional extensions or plugins and some tweaks in the project configuration our IDEs will support the scripting language. Why should we accept code editing like in the early 1990s?

Using NATstyle from the commandline

Last year a new client contracted me to help them make their development process, their development environment and tooling "state of the art" (aka up do date). From the coding perspective that included - among other things - using a decent IDE, writing unit tests, having some Continuous Delivery pipeline in place and using static code analysis. I thought that it should not be too difficult, right? It was, but that is not the goal of this article. Today I want to describe the static analysis features and how to use it.

NATURAL
The code base was NATURAL. NATURAL is an application development and deployment environment using a proprietary language maintained by Software AG. It was created in 1979. André describes it as Cobol's ugly, brain-damaged, BABBLING IN ALL-CAPS - but regrettably still healthy and strong - cousin.. ;-) Well, it is a procedural language structured using modules, subroutines and functions. It feels as old as it is. For example, module names are limited to eight (8!) characters and there is no way to structure the modules further. An enterprise application might contain 5.000 to 10.000 modules. To me NATURAL feels much like early Pascal: units (modules), subroutines and functions together with the eight characters DOS name limit. I really liked Pascal back then and I do not feel that bad about NATURAL. Probably also because I do not have to use it on a daily basis.

NaturalONE and NATstyle
NaturalONE is the Eclipse-based tool for NATURAL by Software AG. It looks pretty good, similar to what you would expect from Eclipse. It contains the usual features like Outline, Search, Debug and something called NATstyle. Now NATstyle - which probably on purpose sounds similar to the well known Checkstyle - is an utility to define and check the coding standard in your programs. It is used inside NaturalONE, see a NATstyle Demo by Software AG. (For more information refer to the Eclipse/NaturalONE help topic Checking Natural Code with NATstyle.)

Invoking NATstyle from outside NaturalONE
I like static code analysis and consider it a mandatory component of every delivery pipeline. As I said above, I wanted to have some static code analysis and was wondering if I would be able to run NATstyle in the pipeline? I did not find any information on the topic and was forced to experiment. I used NaturalONE 8.3.5.0.242 CE (November 2016) and figured out several ways, which were hardly documented, and might be subject to change. Likely there are other options available in newer versions of NaturalONE and NATstyle.

Invoking NATstyle from command line
NATstyle is just an Eclipse plugin inside NaturalONE. In my edition it is the folder C:\Natural-CE\​Designer\​eclipse\​plugins\​com.​softwareag.​naturalone.​natural.​natstyle_8.3.5.0000-0242. With the proper class path it is possible to invoke NATstyle's main method, which prints help on its usage:

Usage: com.softwareag.naturalone.natural.natstyle.NATstyle [-options]

where options include:
  -projectpath <directory> Specify where to find the Natural project.
  -rootfolder              Library root folder support enabled.
  -c <file>                Specify the configuration file.
  -o <file>                Specify the output file.
  -sourcefiles <srcList>   Specify list of source files to be loaded.
                           separated with ;
  -libraries <libList>     Specify list of libraries to be loaded.
                           separated with ;
  -exclude <libList>       Specify a list of libraries to exclude.
                           separated with ;
  -p <directory>           Specify where to find additional packages.
  -help                    Display command line options and exit.

Make sure the following classes can be found in your class path:

com.softwareag.naturalone.natural.auxiliary
com.softwareag.naturalone.natural.common
com.softwareag.naturalone.natural.parser

Nice, that looks like the developers from Software AG prepared NATstyle to be called stand-alone in my pipeline. +1. All the necessary classes can be found in the following plugins of Eclipse/NaturalONE:

• Folder com.softwareag.naturalone.natural.natstyle_8.3.5.0000-0242
• Jar com.softwareag.naturalone.natural.auxiliary_*.jar
• Folder com.softwareag.naturalone.natural.common_8.3.5.0000-0242
• Jar com.softwareag.naturalone.natural.parser_*.jar
• Jar org.eclipse.equinox.common_*.jar

These are all bundles defined by Eclipse/OSGi. The list of dependencies of these bundles also contains the following Jars

• org.eclipse.swt.win32.*.jar
• org.eclipse.ui.console_*.jar

which (in my experience) are not needed to use NATstyle. NATstyle does not use any other OSGi features and we can set the needed class path by hand. Here is an example how to do that in the Windows shell when NaturalONE is installed:

set N1=C:\Natural-CE\Designer\eclipse\plugins
set CLASSPATH=^
%N1%\com.softwareag.naturalone.natural.natstyle_8.3.5.0000-0242;^
%N1%\com.softwareag.naturalone.natural.auxiliary_8.3.5.0000-0242.jar;^
%N1%\com.softwareag.naturalone.natural.common_8.3.5.0000-0242;^
%N1%\com.softwareag.naturalone.natural.parser_8.3.5.0000-0242.jar;^
%N1%\org.eclipse.equinox.common_3.6.200.v20130402-1505.jar

java com.softwareag.naturalone.natural.natstyle.NATstyle %*

Download the source and see bin/run_natstyle for the script I used to run NATstyle directly. Copying the needed folders and Jars to another machine works well. I recommend to create a Jar from the folders before copying them around. bin/_create_jar shows how to do that. Make sure not to include META-INF\*.RSA or META-INF\*.SF into the new Jars because the checksums do not match. bin/copy_jars does the actual copying into the lib folder.

Invoking NATstyle from Ant
NaturalONE supports Ant for some automation tasks. There is no specific NATstyle Ant task, but calling it from Ant is straight forward. If the path <path id="natstyle.classpath"> is set up to contain all the Folders and Jars, NATstyle is executed via Ant's java task:

<java taskname="natstyle"
      classname="com.softwareag.naturalone.natural.natstyle.NATstyle"
      dir="${basedir}" fork="true" maxmemory="128m"
      failonerror="true"
      classpathref="natstyle.classpath">
  <arg value="-projectpath" />
  <arg value="../NatUnit/NatUnit_L4N" />
  ...
</java>

A new JVM needs to be forked because NATstyle calls System.exit() at the end. All parameters are the same as for the command-line invocation and are passed in via the <arg /> element. In the source, see ant/ant_NatStyle.xml for the complete Ant script.

Reporting
NATstyle writes an XML report of all files if worked and found rule violations. The structure is

<NATstyle>
  <file type='...' location='...'>
    <error severity='...' message='...' />
    ...
  </file>
  ...
</NATstyle>

Inside the folder of NATstyle (i.e. com.softwareag.naturalone.natural.natstyle_8.3.5.0000-0242) there is a subfolder NATstyle. It contains sample files. Using the provided NATstyleSimple.xsl the report XML (e.g. NATstyleResult.xml) can be transformed into a human-readable format (e.g. NATstyleSimple.html). When using NATstyle stand-alone, you might want to copy this folder as well. XSLT transformation task is part of Ant and converts the XML to HTML.

<xslt basedir="${basedir}"
      destdir="${basedir}/report"
      style="${basedir}/NATstyle/NATstyleSimple.xsl">
  <include name="NATstyleResult_*.xml" />
</xslt>

See ant/ant_NatStyleResultToHtml.xml for the complete Ant build script to convert the NATstyle result XML into a readable HTML report. Using your own stylesheet (.xsl) allows you to customize the report. See how to convert the report into more complex formats.

Next time I will show you how to write your own NATstyle rules.

Complete Cofoja Setup Example

Design by Contract
Have you heard of Design by Contract (short DbC)? If not, here is a good introduction from Eiffel. (In short, Design by Contract is one of the major mechanisms to ensure the reliability of object-oriented software. It focuses on the communication between components and requires the interactions to be defined precisely. These specifications are called contracts and they contain Preconditions, Postconditions and Invariants. Unlike using assertions to ensure these conditions, DbC considers the contracts important parts of the design process which should be written first. It is a systematic approach to building bug-free object-oriented systems and helps in testing and debugging.)

Cofoja (Contracts for Java)
Cofoja is a Design by Contract library for Java. It uses annotation processing and byte code instrumentation to provide run-time checking. It supports a contract model similar to that of Eiffel, with added support for a few Java-specific things, such as exceptions. In Cofoja, contracts are written as Java code within quoted strings, embedded in annotations. Here is some sample code (derived from lost icontract library): A basic stack with methods to push, pop and to see the top element.

import java.util.LinkedList;
import com.google.java.contract.Ensures;
import com.google.java.contract.Invariant;
import com.google.java.contract.Requires;

@Invariant({ "elements != null",
             "isEmpty() || top() != null" }) // (1)
public class CofojaStack<T> {

  private final LinkedList<T> elements = new LinkedList<T>();

  @Requires("o != null") // (2)
  @Ensures({ "!isEmpty()", "top() == o" }) // (3)
  public void push(T o) {
    elements.add(o);
  }

  @Requires("!isEmpty()")
  @Ensures({ "result == old(top())", "result != null" })
  public T pop() {
    final T popped = top();
    elements.removeLast();
    return popped;
  }

  @Requires("!isEmpty()")
  @Ensures("result != null")
  public T top() {
    return elements.getLast();
  }

  public boolean isEmpty() {
    return elements.isEmpty();
  }

}

The annotations describe method preconditions (2), postconditions (3) and class invariants (1). Cofoja uses a Java 6 annotation processor to create .contract class files for the contracts. As soon as Cofoja's Jar is on the classpath the annotation processor is picked up by the service provider. There is no special work necessary.

javac -cp lib/cofoja.asm-1.3-20160207.jar -d classes src/*.java

To verify that the contracts are executed, here is some code which breaks the precondition of our stack:

import org.junit.Test;
import com.google.java.contract.PreconditionError;

public class CofojaStackTest {

  @Test(expected = PreconditionError.class)
  public void emptyStackFailsPreconditionOnPop() {
    CofojaStack<String> stack = new CofojaStack<String>();
    stack.pop(); // (4)
  }

}

We expect line (4) to throw Cofoja's PreconditionError instead of NoSuchElementException. Just running the code is not enough, Cofoja uses a Java instrumentation agent to weave in the contracts at runtime.

java -javaagent:lib/cofoja.asm-1.3-20160207.jar -cp classes ...

Cofoja is an interesting library and I wanted to use it to tighten my precondition checks. Unfortunately I had a lot of problems with the setup. Also I had never used annotation processors before. I compiled all my research into a complete setup example.

Maven
Someone already created an example setup for Maven. Here are the necessary pom.xml snippets to compile and run CofojaStackTest from above.

<dependencies>
  <dependency> <!-- (5) -->
    <groupId>org.huoc</groupId>
    <artifactId>cofoja</artifactId>
    <version>1.3.1</version>
  </dependency>
  ...
</dependencies>

<build>
  <plugins>
    <plugin> <!-- (6) -->
      <artifactId>maven-surefire-plugin</artifactId>
      <version>2.20</version>
      <configuration>
        <argLine>-ea</argLine>
        <argLine>-javaagent:${org.huoc:cofoja:jar}</argLine>
      </configuration>
    </plugin>
    <plugin> <!-- (7) -->
      <artifactId>maven-dependency-plugin</artifactId>
      <version>2.9</version>
      <executions>
        <execution>
          <id>define-dependencies-as-properties</id>
          <goals>
            <goal>properties</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

Obviously we need to declare the dependency (5). All examples I found register the contracts' annotation processor with the maven-compiler-plugin, but that is not necessary if we are using the Maven defaults for source and output directories. To run the tests through the agent we need to enable the agent in the maven-surefire-plugin (6) like we did for plain execution with java. The Jar location is ${org.huoc:cofoja:jar}. To enable its resolution we need to run maven-dependency-plugin's properties goal (7). Cofoja is build using Java 6 and this setup works for Maven 2 and Maven 3.

Gradle
Similar to Maven, but usually shorter, we need to define the dependency to Cofoja (5) and specify the Java agent in the JVM argument during test execution (6). I did not find a standard way to resolve a dependency to its Jar file and several solutions are possible. The cleanest and shortest seems to be from Timur on StackOverflow, defining a dedicated configuration for Cofoja (7), which avoids duplicating the dependency in (5) and which we can use to access its files in (6).

configurations { // (7)
  cofoja
}

dependencies { // (5)
  cofoja group: 'org.huoc', name: 'cofoja', version: '1.3.1'
  compile configurations.cofoja.dependencies
  ...
}

test { // (6)
  jvmArgs '-ea', '-javaagent:' + configurations.cofoja.files[0]
}

Eclipse
Even when importing the Maven project into Eclipse, the annotation processor is not configured and we need to register it manually. Here is the Eclipse help how to do that. Fortunately there are several step by step guides how to set up Cofoja in Eclipse. In the project configuration, enable Annotation Processing under the Java Compiler settings.

Although Eclipse claims that source and classpath are passed to the processor, we need to configure source path, classpath and output directory.

com.google.java.contract.classoutput=%PROJECT.DIR%/target/classes
com.google.java.contract.classpath=%PROJECT.DIR%/lib/cofoja.asm-1.3-20160207.jar
com.google.java.contract.sourcepath=%PROJECT.DIR%/src/main/java

(These values are stored in .settings/org.eclipse.jdt.apt.core.prefs.) For Maven projects we can use the %M2_REPO% variable instead of %PROJECT.DIR%.

com.google.java.contract.classpath=%M2_REPO%/org/huoc/cofoja/1.3.1/cofoja-1.3.1.jar

Add the Cofoja Jar to the Factory Path as well.

Now Eclipse is able to compile our stack. To run the test we need to enable the agent.

IntelliJ IDEA
StackOverflow has the answer how to configure Cofoja in IntelliJ. Enable annotation processing in Settings > Build > Compiler > Annotation Processors.

Again we need to pass the arguments to the annotation processor.

com.google.java.contract.classoutput=$PROJECT_DIR$/target/classes
com.google.java.contract.classpath=$M2_REPO$/org/huoc/cofoja/1.3.1/cofoja-1.3.1.jar
com.google.java.contract.sourcepath=$PROJECT_DIR$/src/main/java

(These values are stored in .idea/compiler.xml.) For test runs we enable the agent.

That's it. See the complete example's source (zip) including all configuration files for Maven, Gradle, Eclipse and IntelliJ IDEA.