IBM Champion!

IBM announced the nominations for IBM Champions 2017 and I’m very happy to say that I was nominated.

It’s wonderful to have this sort of recognition and suddenly being part of a group who have made huge contributions to the IBM ICS Community of the years is humbling.

I strongly suspect that my nomination is linked to my co-organising the Swiss Notes User Group, so I’d like to mention at this point my co-conspirators, without whose work SNoUG wouldn’t happen:

Diana Leuenberger, Susanne Schugg and Helmut Sproll

And also many thanks to those that have contributed behind the scenes to make SNoUG a success – I think in particular here of Paul Harrison, with his wonderful AgendaWALL and AgendaPAD, as well as the very talented LDC Via Group, who help me whenever I’m stuck.

I’d also proud of strengthening the Swiss IBM Champions group – showing that plucky little Switzerland can pack a good punch!

From 3000 milliseconds to 200 to open a page: Speed up your Notes Client applications by discovering their bottlenecks.

I had inherited a template for a database which was taking 3 seconds to open any sort of document. It soon got to be irritating, and I noticed that the little lightning bolt at the bottom left side of the Notes client was flashing wildly, so I fired up the Notes RPC Parser to try and see what was all the traffic about.

The culprit was soon found: it was a subform containing buttons whose label was dynamically computed depending on the user’s desired language. And, unfortunately, it was making an @DbLookup for every single button, and an uncached one at that. If you look at the screenshot, you’ll see that an @DbLookup generates 3 NRPC Calls, for a total of 50ms – and this was done 30 times each time a document was opened.

That, and another issue that I corrected, meant that I could massively improve the performance of the application for only about half a day’s worth of work.

screen-shot-2016-11-30-at-16-58-34

Feel free to use the tool, or if you can’t be bothered, hire me to solve your Notes Client performance issues.

P.S. I’ve just released a new version of the tool Notes RPC Parser, which you can download here:

Host your migrated Domino Data in Switzerland at ch.ldcvia.com

LDC Via is a dream destination for your Domino data. If you are looking to decommission your Domino infrastructure, LDC Via offers you a Platform-as-a-Service (PaaS) solution for hosting your migrated Domino data. Once the data are transferred, they are available using the very modern REST interface which all modern Web applications (including native mobile applications) can easily access.

Certain Swiss customers (such as local governments) are not allowed to host their data outside of Switzerland.

For these customers, and those of you who wish to avoid any interaction with US companies because of legal concerns, we have set up a Swiss instance of LDC Via, which is immediately available under

ch.ldcvia.com

I invite you to start exploring the service! Please contact me at andrew.magerman@magerman.com or +41 79 831 83 13 if you have questions about the service.

The pricing for the Swiss service follows the tiers defined at ldcvia.com/prices.html

Category monthly price
Basic Free
Standard 400 CHF per month
Pro 1’000 CHF per month
Enterprise 2’000 CHF per month

all prices excluding VAT.

Why do Software Developers always squawk when inheriting a foreign codebase?

The first thing that Software Engineers do when they inherit a codebase that they haven’t written is to squawk loudly.

This is terrible! That’s not the way to do it! We need to re-write this completely!

I recently had an experience when a project manager suddenly lost his developer and asked me to take over. ‘Here is the list of requirements from the customer, they’re just small changes’.

I said ‘OK, I’ll give this a look, doesn’t look too difficult‘. And then I started looking into the code, and I started grumbling more and more. There were no comments. No source control. No test code. The code I had in front of me what not working correctly, and I had no access to a codebase which was working. And within the code, there were so many times when I was thinking ‘Oh no, why would you do that? I’d really do it differently’.

So I promptly squawked. And the project manager expressed his incredulity. ‘It’s always the same with developers! They always complain of code that they haven’t written themselves!’.

So, here’s a little guide to explain why this happens.

We’re not masons.

That’s the crux. The job of building walls is well defined, and if a wall is not complete, you can hire another mason to finish the job. He’ll come to the site and the tools that he sees are familiar and he can start right where the previous person stopped. Easy.

That’s not how it works with software. There is a wide range of ways you can solve something, the wider since we basically can write your own tools. Another persons’ building site is going to be completely unfamiliar.

So here are my insights for non-developers, in the hope that you will get over the impression that we are whiners, and understand that software development is not masonry.

Software is easier to write than to read.

After about six months, even the code you have written yourself becomes more difficult to read. A good developer will be writing his code not just well enough for the machine to understand (bad developers stop at that point), but well enough for humans to understand.

This is a pretty accurate picture of what it’s like reading other people’s code from somebody else:

Software can be really badly written, and you can’t see it from outside.

This rejoins the points I was making in a previous post. Here we’re talking about maintainability, the ease with which a particular codebase can be modified.  Here are some best practices which you should insist that your developers follow:

  • One is to separate the responsibility of code into little independent boxes, which only do one specific thing. If you need to change something, there is only one place to look for. This is called modularity and follows the concept of separation of concern.
  • Another one is having Source Control. This shows all the changes that have been done on the codebase over time, including who did it, and enables us to revert code to what it was in the past (when a bug has been inserted, for instance.)
  • Automated tests make every small bit of the code run through its logic, often by trying to make it break. There is one thing that we software developers are scared of, and which happens a lot: we change a bit of code here and then notice that loads of other bits of code are no longer working. Automated tests enable you to be able to make bold changes because you immediately get feedback on whether you’ve broken something.
  • Finally, documentation really helps. And please note that I’m not talking about some separate Word Document that was once done for the client, I’m talking about comments explaining why things were done in a particular way embedded within the code.

If a software developer inherits a codebase which didn’t follow these guidelines, then that code is going to be a pain to maintain and to change. And as a developer, it’s a large source of frustration because we are in effect far slower at delivering changes than our predecessor, and our customer is not going to be impressed.

So, if you are a customer, request that these basics are followed, or else you’ve got a bad case of developer lock-in, where Bob is the only person who can maintain the code.

In conclusion:

If you are a customer and switch developers or provider for a project, you’ll have to factor in some budget and time just for the new developer to get acquainted with the project. You’ll also see that it will initially take longer to effect changes, depending on how well and maintainable the software was written in the first place.

 

LDC Via is a dream destination for migrated Notes applications

It’s becoming increasingly clear that IBM Domino/Notes has reached its software end-of-life.

IBM has not announced it, but the proof is in the pudding, and the pudding says no new releases since 3 years, the pudding says minimal IBM resources earmarked for development (almost all the exciting new stuff has been a community open-source work, many thanks to the heroes at OpenNTF), the pudding says outdated development tools, circa 8 years old. I’m writing this article on the unconfirmed assumption that IBM has no interest in preserving either the Notes Client, the Domino server, and the nsf storage file.

It’s time to start thinking about exit strategies.

For companies that decided to move off from Notes/Domino, after the easy part of migrating mail-files and changing mail servers, they were usually left with a plethora of functioning, sometimes business critical applications, for whom the cost of a complete re-write in another platform was prohibitive (more on that later – it’s still the case). The exit costs are high, and most organisations decided to keep their Notes infrastructure running, in some sort of compromise co-existence scenario, often with the fat clients hidden away or available only to a small subset of users.

But now that IBM is (apparently) pulling the plug, the ‘application migration’ question comes up again. The costs are still high. Sorry. Why?

The ugly secrets of most of the Notes applications that I’ve run across are

  1. They are badly documented (a crime which presumably is prevalent in most in-house developments, as opposed to professional software houses)
  2. The development tools invited developers to work in a quick-and-dirty way, with no structure, producing fast initial results but subsequently slow updates (This was unfortunately also the case for XPages applications).
  3. The absence of any way to do systematic testing also led to an accretive style of coding, where one only adds new stuff, and getting rid of old stuff is so dangerous that nobody dared to do it. This also means that, in a historically grown application, there is going to be a lot of cruft within the application – it’s impossible for a developer to know which are the bits that are needed, and which are obsolete.

You’ll need the following for a successful application re-write:

  1. An application owner from the business side who has followed the history of the tool and will be able to tell the developer which bits are used and which are obsolete
  2. A developer that can ‘read’ the old Notes application, not only in terms of code but also in terms of data structure and security.

Once you have these two sine qua nons, the question arises as to which destination should you choose. Of all the options, my vote goes to LDC Via.

LDC Via makes the re-write job a lot easier. Why?

  1. The data structure is the same, so the accessing logic in the new application doesn’t have to change.
  2. The document-level security upon which most Notes applications run is still available.
  3. The data is accessible using a modern interface called REST which is modern, simple to understand, use and test.

This means that you can really concentrate on business logic and user interface and don’t have to spend time translating the original ‘Notes’ logic.

I’m looking forwards to my first application migration projects working with LDC Via, and I’m also proud to announce that my company has become a technology partner of LDC Via.

Please feel free to contact me if you are interested.

Software is an iceberg. You’ll get hurt by the submerged parts.

Software Quality is an iceberg.

The tip of the iceberg is the small part sticking out that everybody can see. A non-IT person judges software with a far smaller subset of criteria than an IT professional: Functionality, performance, and usability.

To a normal user, the interface is the software. I regularly have to choke off exasperated comments when a customer tell me ‘but it’s only one extra button‘.

Lurking beneath the murky waters are those criteria which only IT professionals see: reliability, maintainability, security.

End-users should not be expected to see these ‘underlying’ aspects, but if the decision-makers (i.e. the ones with the purse string) don’t see under the water, you’re doomed to produce bad software. Tell-tale signs are:

  • There is no budget for IT maintenance
  • The only time fresh money is given is for new functionalities – money is never given for a functionality-free maintenance
  • Projects get stopped as soon as the software is barely out of beta testing. Finally, as a programmer, you’ve produced software that works, but in your mind, this is just a first version. There are lots of bits and bobs of useless code left in your software, debug code that writes to system.out, the object and data structure is a mess and could be optimised, you’ve yet to apply a profiler to your code to see where are the bottlenecks. And then the project gets stopped: ‘it works well enough’. That is very upsetting, and possibly one of the main reasons that in-house IT jobs are far less attractive than IT jobs within software companies.

If you recognise yourself as a company, then you should give more power to the IT department. Too many companies treat IT as a necessary, needlessly costly evil. IT should be an essential, central part of your company strategy. There is no other region of a company’s activity which gives you more of a competitive edge.

Too many companies (and I am looking at banks and insurances in particular) have failed to see that they are, essentially, a software company. You need to have a CIO who understands his trade, and accept that he is going to spend money for stuff which doesn’t make sense to you, or at least doesn’t seem to change anything.

If you are only ready to pay for the IT stuff which is visible – more functionality, for instance, but no money for a refactoring exercise which would add stability but not change the functionality, the accumulated technical debt is going to bite you at some point. It might be that an application gets exponentially difficult to change, or it might be that you have a massive data loss that you can’t recover from because the backup software was not up-to-date and nobody did an actual restore as part of regular maintenance.

If you are a developer, and you find yourself in such an environment, you’ll have to go guerilla. You’ll need to surreptitiously improve your code whenever you have the opportunity. Try to leave any code you touch a little bit better than before. Notice duplications and remove them. If you didn’t understand part of a code, then document it after you have understood it. If variable names are unclear, then rename them to something better. It’s an thankless occupation, because there is the risk that you break things that were working, and nobody will really care that you reduced the codebase by half, but it will help you progress and it will increase the quality of your work.

IT is surprising in that often the perception of quality is completely at odds with real quality, because of this iceberg effect. I once used to work in an environment where colleagues were doing terrible things – changing code directly in the production environment, and only doing quick-and-dirty cosmetic patches, but were actually perceived by the customers as delivering excellent quality, because the response times were so fast.