Engage 2015 as engaging as ever

Engage is the most successful LUG in Europe, and as usual I am slightly bewildered by how Theo Heselmans, our gracious host, manages to pull it off. The venue was lovely, the opening session room stunning, and the content was very high quality. I really enjoyed meeting many community members whom I had only seen online, including a couple of my Stackoverflow saviours (Per, nice to have met you!)

The city of Ghent itself was a nice surprise. The inner city is full of history, with many old buildings harking back to a more prosperous past, and a surprising number of churches. I had a little walk to the north of the city, though, and it’s obvious that the city went through an industrial phase which it got out of and has not really recuperated from.

Here are my technical take homes from the whole two day session:

Both Ulrich Krause and Frank van den Linden independently confirmed that they didn’t like the new ‘Java’ element and found the oldskool WEB-INF folder stabler.

Theo Heselmans presented some of the Javascript frameworks he’s been using; I knew Bootstrap and Backbone; he recommended Ratchet and Knockout as well. Also, if you want to store local stuff, the way to go nowadays is no longer cookies but the manifest or local storage.

John Daalsgard had a good session explaining the Domino REST API; I learned lots of stuff but was sort of disappointed that authentication was not really talked about. Most of the examples were using anonymous access, and authentication is still not really an easy thing to do. Paul Harrisson, who did the local web application for engage, pointed me to his blog entry about authentication. I’ve been working with Julian Buss’ framework DominoToGo and I was initially under the impression that the REST Services introduced in 9.0.1 would mitigate its usefulness but I’m coming to the conclusion that as soon as you get out of the demo-cases ‘simple text’ and ‘anonymous access’ things start getting complicated using REST, i.e. one has got to start coding things oneself.

One of the most interesting sessions was the one on GIT done by Martin Jinoch and Jan Krejcarek. Martin was very stern and he endeavoured to persuade us to abandon the idea that the source code resides within the NSF and that the Git repository is the backup. Rather, the source code is in GIT and the NSF is just a throwaway, last-minute build construct. I almost broke in tears. Martin also admonished us to turn everything off that automatically builds, including the nsf to on-disk-project sync.

I was also relieved to hear that other fellow developers were irritated by ‘false positives’, i.e. files that have been touched, and therefore appear in the staging area of git, but whose code has not been practically modified, and therefore are really cluttering. There is a project called DORA which alleviates this, but it only works if one starts the project with it. Implementing it midway is bad, apparently (thanks to Serdar).

The London Developer Co-op was there in force, with a stand even, and showed us a very polished product for data exporting. I can see use cases if customers just want to store their data somewhere else, to finally kill off the remnants of their Domino infrastructure, but the fact that the business logic does not get exported will still represent a large exit barrier.

Mark Leuksink and Frank van den Linden introduced me also independently to bower, a package manager that manages the javascript library dependencies automatically for you. The idea here, if you’re doing an XPages project, to have bower point at your ods structure and do the updates here. You’ll need to press F9 in the package explorer before syncing the project.

In the mindblowing categories, Nathan Freeman showed the Graph construct he has made available within the OpenNTF Domino API. Documents stored in nsf without views? That’s just weird. Possibly illegal. And whereas I can see obvious advantages in terms of speed when the data structure is already known in advance, especially for transversal, multi-layered searches like ‘show me the persons who know the persons I know’, I’m not sure how the Graph concept would deal with ad-hoc requests, or with a change in the underlying data structure. I would really like to see what sort of measurements one can make as to the performance of data writing and reading, especially in large numbers. The demonstrations as well were built from scratch, and worked well, and I’d be very interested to see what happens when one takes an existing data landscape and ‘graphs’ it.

The final session I attended was from Paul Withers and Daniele Vistalli. Paul presented the newest possibilities of the next version of OpenNTF Domino API. They are introducing a concept of metaversalID which is a combination of database replicaID and Document Universal ID, and apparently the code has been made Maven-compatible. It looks like we will have, in conjunction with Christian Güdemann’s work on an eclipse builder, soon a system where we can start thinking of continuous builds. We’ll be big boys, then, finally.

Daniele introduced the Websphere Liberty Server. I had dismissed the Websphere server as a huge, lumbering IBM monster but apparently the Liberty Server is small and lightweight. And then, doing some magic, Paul and Daniels made the Liberty Server behave just like a Domino server. The demonstration was still very much in beta stage, and I’m not clear as to the implications of this tour de force. But it might be a game-changer.

my non-technical take homes:

When travelling, bring two phone chargers. With the iPhones losing juice so quickly, losing your charger leaves you strangely vulnerable and incommunicado. Thanks to Ben Poole for letting me load up at the LDC Via stand.

It is unwise to start debating with Nathan Freeman at 2.30 in the morning after everyone else has been kicked out of the hotel bar, and Nathan has a bottle of tequila in an ice bucket.

Loop elegantly through a JavaScript Array

I’ve been reading JavaScript Enlightenment to try and understand the language. There is beauty, and power, hiding behind the covers of JavaScript, but I haven’t clicked yet. I still don’t really get prototypal inheritance and what ‘this’ really means. There is power in understanding closure and scope, too, I am sure. This small book is recommended to see where the beauty is.

20943392

I’d like to share one bit of code which I found elegant: looping through an array:

var myArray = ['blue','green','orange','red'];

var counter = myArray.length;

while (counter--) {
    console.log(myArray[counter]);
}

It relies on the fact that integer values that are not 0 or -0 are ‘truthy’.

Visually document the dependency tree of Script Libraries in your database with one click

The two ugly sisters of software quality are documentation and testing. Both are sure-fire indicators of how easy a piece of software is going to be to maintain. They are not sexy, though, and most customers or end users don’t care about them.

Nor pay for them.

Bad documentation and missing testing routines makes their hit on quality insidiously, as codebases become incrementally more complex, more difficult to decipher, and changes to the software become, apparently unexplainably, more and more costly, and risky, until it reaches the ‘legacy’ status – change me at your own risk and peril, bwoa-ha-ha-ha!

Manually generated documentation is useless. It’s almost immediately out of date. The only real documentation is the code itself, and that’s why any kind of documentation should be automatic, taken from the actual application. JavaDoc is superb, as was Mikkel Heisterberg’s lsdoc.org (which is sadly not online anymore – please join me in bribing Mikkel to reinstate it).

The Domino Designer Plug-in Dependency Documenter is a modest contribution to help you document IBM Notes applications. Once the plug-in is installed, a click of the button will make a diagram showing how the database’s script libraries depend on each other.

The practical application is to help fight the compilation problems that sometimes happens when Notes gets confused about time-stamps (Generic LSE Failure (no information), anybody?), and a sure way to work this around is to re-save the script libraries in reverse order of dependency. The diagram helps you to find out where to start.

Final Result

Please note that the heavy lifting of the diagram generation is done by Graphviz, which you’ll need to install separately on your development machine. Graphviz is a free, open source library which does a remarkably good job of making clear diagrams based on a simple inputted text file.

The plugin itself is still somewhat rough around the edges, and I’d welcome any kind of feedback or any feature requests you might find useful.

All of this would not have been possible without the previous work of Karsten Lehmann (mindoo), René Winkelmeyer (midpoints), Ulrich Krause (BCC Consulting) and Ralf Petters (Artweger GmbH), all of whom contributed fantastic presentations and blog entries on creating Notes Plug-ins. A special thanks to Ralf, who held my hand whilst I was trying to set up the development environment for plug-in development.

Tim Tripcony’s incinerate function elegantly batch recycles Domino Objects in Java

I recently was surfing through Stackoverflow and I hit a response from Tim Tripcony on this post: What is the best way to recycle Domino Objects in Java Beans

He published a small helper function to recycle an arbitrary number of Domino objects. It’s so simple, and so well written, that I thought it deserved a post of its own. Here it is in all its glory:

private void incinerate(Object... dominoObjects) {
    for (Object dominoObject : dominoObjects) {
        if (null != dominoObject) {
            if (dominoObject instanceof Base) {
                try {
                    ((Base)dominoObject).recycle();
                } catch (NotesException recycleSucks) {
                    // optionally log exception
                }
            }
        }
    }
}

And it’s used like this:

   Database wDb = null;
    View wView = null;
    Document wDoc = null;
    try {
// your code here
    } catch (NotesException ne) {
 // do error handling
    } finally {
        incinerate(wDoc, wView, wDb);
}

I find the function elegant in many ways; the function and the variables are clearly named, Tim uses variable-length argument lists (thats the Object… part with the ellipsis), the new for-each iteration, plus there is some humour. Thanks, Tim.

SSJS is an abomination born out of IBM’s misguided condescension

Scott Souder was recently extolling the skill of the IBM developers and added as an aside “these are not your average domino developers”. I flinched at the statement and thought defensively “does he mean me?”

The more I think of it, though, and it becomes obvious that this is a central assumption that affected the design of XPages. It’s even there, in the introduction to the mastering XPages book.

I would like to humbly submit to IBM that domino developers nowadays are not an unskilled community, and that its modeling of who domino developers are needs some updating, in the hope that this will positively affect further design decisions. We’re not the Samanthas of the programming world.

It’s immediately obvious at any notes gathering that we are not the youngest community – I rarely see anybody in their twenties, for instance. It’s therefore obvious that we are all experienced. Even those without a computer science background (and this applies to me) has learnt, the hard way, that it’s the maintenance and debugging of code which is complex, not the writing. I learned, the hard way, that copy-pasting code is a shortcut which kills you a couple of years later. I learned that structuring code cleanly, with small functions doing a single thing, ends up being far easier to maintain than two thousand lines of procedural code. Making the switch from procedural code to object oriented code helped tremendously, along with the concept of separating business logic from the displaying logic.

XPages was confusing to me at the beginning. It seemed like a step backwards. Squirreling bits of JavaScript into the XML code, into what clearly should strictly be a presentation layer, mixing business logic with presentation logic, after spending years moving code from buttons and hotspots in forms into script libraries? It doesn’t make sense – until you understand the central assumption of IBM, which was that we would not get Java. Too difficult. The dumbo domino developers will never get it. They only get quick-and-dirty RAD.

Then, in the course of my XPages learning curve I stumbled into managed beans, and everything clicked into place. This is where the beauty lies, this is where I can put my layered, easily debuggable, easily testable code. It’s easy to bind to the XML source, and the Expression Language is concise and powerful. And yet these sections are under-documented, the know-how gleaned from stack overflow and numerous blog entries on the subject made by us, dummy domino developers.

In the face of the obvious scarcity of resources that IBM has devoted to the Notes platform, it is such a crying shame to have invested so much into SSJS and the whole formulation of formula language as JavaScript functions. (An impressive achievement, but a misguided one in my opinion).

The resources could have been better spent elsewhere. Learning java is but one of the challenges of learning XPages, and it’s a challenge for which there are tons of documentation.

There’s often a picture made about misguided security in IT – a brand-new lock on a rotten door doesn’t increase your security. Here there is a parallel with misguided condescension – Removing Java out of the learning curve is just a small contribution to making XPages simpler. There’s a plethora of new stuff that we must learn, we must get our heads around conflicting terminology (e.g. Notes View, JSF View, Eclipse View), a lot of functionality comes from knowing that the CSS class you need is called ‘lotusInlinelist lotusUtility’, there are missing tools such as JUnit, refactoring, renaming so you end up having to do a lot of things manually…

SSJS annoys me because of the underlying assumption – I am dumb.

SSJS also annoys me architecturally. It breaks the MVC model pretty badly. It probably also affects performance and scalability (although I have no figures to support this)

And it’s also a misnomer: It’s not at all JavaScript being run on the server (like node.js would do, blindingly fast on top of Google’s V8 engine). At runtime it’s really running as Java. Guess why it takes so long to build your projects? Our poor computers have to interpret the ‘JavaScript’ and translate this into Java code. And your guess is as good as mine when it comes to the new versions of JavaScript – will IBM update too, to ECMA 5 Strict, say?

My tip: circumvent SSJS as soon as you can and go straight to Java.

Powerful searches in DDE: Regex for ‘all bar that don’t start with foo’

Regexes are wonderful. There’s a learning curve, and they are very cryptical to be able to read, let alone write, but once you get the hang of it they are wonderful.

I was doing some refactoring and I wanted to see all instances of a function called ‘AddButton’ – and my results got saturated with another function ClickAddbutton.

Which Regex to use to ask for ‘Show me all AddButtons but ignore ClickAddButton’?

The trick here is to use negative lookbehind:

(?<!textIDontWantPrefixed)TextIAmLookingFor

so in my example this was what to input in the search box in DDE:

(?<!Click)AddButton

This is the way it works:

1. Search for ‘AddButton’

2.When you’ve found ‘AddButton’, look if there is a naughty prefix (‘Click’) which invalidates the search.

This is the lookbehind part. It’s slightly confusing because a lookbehind is going to look at the bits to the left of the word, which one would typically call ‘before’. The regex convention is that forwards is the direction of parsing and behind is the other direction.

3. if the naughty prefix isn’t there, return a match.

That’s the negative part.

Voilà!

P.S. an excellent tool for Regexes is RegExBuddy (Windows only though). Really recommended, saved me loads and loads of time. I wouldn’t do regexes without it.

Leave Code in the state you would like to find it in.

It’s been one of my principles since I started being an independent developer. Much of my work is maintaining old codebases, and there are many horror stories that I could recount.

The temptation is there to do a quick-and-dirty modification, a quick patch, but one should resist the temptation.

Customers generally do not understand the concept of refactoring and I have to this date never been able to persuade any customer of parting with cash for refactoring. Arguments about less work down the line just fall on deaf ears.

So the refactoring is included in my normal style of coding – on-the-fly refactoring, if you will. I generally find that it’s well worth refactoring badly written code as you are trying to read it.Renaming variables is my first step at reading the code, then creating subfunctions that have a clear purpose in separate functions. If the code is complex I’ll generally do the step of converting it to object-oriented code. It might seem like a waste of time or even overzealousness, but on the long run it’s well worth it, also for your client, for his maintenance costs are going to go down.

So, whenever I am within a function that I need to update or correct, I make sure that I leave the code in a state that won’t make me cringe. This usually means renaming variables (naming them for what the content is, not the container – i.e. docPurchaseOrder instead of doc), adding error trapping, and deleting stuff.

I love deleting stuff. The fewer design elements, the fewer functions there are, the better, if you ask me.

In the past, if you were a Notes developer without access to third-party tools, coding was very much an accretive process. Need a lookup view? Create a new one. Need a slightly different function? copy-paste. Since we had no visibility about dependencies, it was really dangerous to remove things – so the convention was just to add things.

Over the years this results in a terrible mess.

Thankfully there are some tools one still can use. One of my favourites is Teamstudio’s lscalls, a small exe that lists out functions that are no longer called by any other function. This is a really powerful and safe way to identify functions ready for deletion.

Teamstudio no longer posts it on their website (too legacy-ish, presumably) but they will send you the file if you e-mail them and ask nicely.

Just for me, here is an example of the syntax.

C:\Program Files (x86)\IBM\Notes>lscalls “C:\Program Files (x86)\IBM\Notes\Data\Development\eFSR\eFSR3_0\Efsr_(3_0)_Field_Service_Local_Dev.nsf” -o “c:\temp\lscallsoutput.csv”

You will need to place lscalls.exe in your notes program directory.

A little caveat: if a function is shared between multiple databases with inherited script libraries, you might want to make a check on all databases sharing the common code.