Notes/Domino

Domino Keep, an modern API for Domino

by

OpenNTF organised a webinar in July 2021 showcasing Domino Keep which we had the pleasure of attending.

Many of the improvements that HCL brought to the Domino/Notes stack has “catching up”: mostly self-evident improvements from accumulated feedback from domino developers.

These improvements, though, have until now been unable to break the “yellow bubble”: Companies still needs extremely specialized developers and admins if they want to change anything or create anything in their Domino infrastructure.

Modern (and by that we mean young) developers have several frustrating hurdles to cross before they can be productive with Domino. The architecture is alien and monolithic. Having to install a Windows machine is an annoyance, but having to use the slow, Eclipse-bloated monstrosity that is Domino Designer, learning which of the many windows contain the setting you want to change is a pain.

Domino Keep tries to remedy this by presenting an external API that is familiar and follows modern standards. Thankfully HCL assigned two of the smartest Domino gurus that we know of, Paul Withers (who is famous for re-writing the lotus.domino package so that we developers could write modern Java in our Domino applications) and Stephan Wissel (whom we had the pleasure of welcoming at our last SNoUG event in 2019), to the task.

Familiarity:

  • output data is always in JSON format
  • Domino Designer installation is not necessary! Possibly the biggest hurdle.
  • the public API is based on OpenAPI 3.0
    • This means that it comes with a fully interactive auto-documentation
    • The interface uses REST Api conventions, for instance an aclEntry is accessed with /acl/entries/{aclEntryName}
  • since all the calls are REST Calls, you can use the command line, or something like Postman, or even do complex testing with Node RED (recommended)

Security:

  • Keep keeps (pun intended) the tried-and-tested security ideas of Notes Domino and won’t let you override settings in the ACLs.
  • Keep uses the modern JWT (JSON Web Tokens) techology for its authentication and authorization – this will be familiar to modern developers.
  • Keep is secure by default, which means that without configuration (done on a configuration database, KeepConfig.nsf on your server), no access is granted.

Running Agents:

You can trigger Agents on your database with a REST call! This is a sine qua non for being able to reuse your business logic, which often is present in the form of back-end agents which perform regular updates on your document.

Not only data but design elements too:

Previous APIs allowed read/write access to data via REST, but Keep also allows you access to the design data. This means that you can create a database, create a view, create a form, with a simple REST call. In practice HCL has added a layer on top of the DXL representation of Domino design elements, which means that you are limited by DXL’s own limitations, i.e. certain details will not be able to round-trip.

This is one of the more seducing points; with an easy way to create a persistent layer for your application, you are effectively inviting modern developers to create new domino databases, not just read data from existing ones.

The ACLs are also open to modification, so you can control access with the usual Domino granularity without having to go into the Domino Administration client.

Conclusion

All in all we’re impressed by the implementation of Keep, it’s modern, familiar for non-notes developers, standards-compliant and is very good at completely hiding the domino-specific technology for newcomers.

This can effectively open up the pool of developers that can do work on your domino infrastructure from a couple of grizzled old hands to any new developer in your team.

As such, we would recommend you to install it prophylactically on your server so that it is already up and running.

Links

Documentation:

https://opensource.hcltechsw.com/domino-keep-docs/

webinar recording :

webinar slides:

Adé Nathan

by

Nathan T. Freeman passed away on 12th April 2021.

He was one of the gurus of the Notes/Domino developer community, very much at the cutting edge of technology and always pushing the boundaries of what was possible with our beloved Notes stack. An early adopter of XPages, he was one of the first to show that one could significantly improve the quality of the XPages applications by concentrating on pure Java Development (which was unfortunately hidden under the covers of XPages), and also one of the first ones to eschew it.

He was one of the founders of OpenNTF, and in a lucky twist of fate his initial were NTF, the file extension for Notes templates, which I always found particularly nifty.

He had an abrasive online personality completely belied by his charm and affability when met in person. I had the pleasure of emptying a bottle of whisky with him on a long night in Eindhoven at the Engage meetup. Eloquent and quick-witted, he was a pleasure to talk to, and an excellent intellectual sparring partner.

Due to unfortunate circumstances his family is in dire straights, I include a GoFundMe link if you would like to contribute.

We will miss you Nathan.

Andrew

Engage 2019 Recap

by

Theo Heselmans and his wife Hilda have once again pulled off a wonderful event in Brussels. The world’s biggest ICS conference attracted attendees from all over the world, from as far away as New Zealand, Japan, the U.S.A and even La Réunion.

HCL continued to impress with its energy and commitment, but in terms of news there was nothing newer than what was divulged at the HCL Milan factory tour.

The sessions which I thought most thought provoking were Knut Herrmanns’ Web-component session, which demonstrated the creation of custom HTML tags (enticingly, he created a new tag <domino-view>). Knut does have a bad habit of saying ‘this is really easy’ at the moment that I am struggling to understand.
Jesse Gallagher suggested compelling ways of recycling skills learned with XPages, since it has become clear, if not officially admitted, that XPages will not receive much investment in the future.

Gab Davis made an enlightening comparison between the on-premise Mail servers Domino and Exchange and I discovered that Microsoft has made a sterling effort at making the local e-mail client experience very configurable, whereas Domino has crafted an experience which is apparently more admin-friendly. This surprised me, as I thought the admin interface for Domino was not first in class. Frank van der Linden made an honest comparison between the currently popular front-end frameworks Angular, React and Vue, and ended up with preferring Angular. There was some huffing and puffing from the back rows from Thilo Volprich (who prefers Vue) and Knut Herrmann (who prefers React)

Daniel Nashed and Thomas Hampel were obviously having fun showing us how to implement a Domino server within a Docker file; they will make the images and the scripts available to us. I imagine this should simply deployment, especially for small contained applications, but I can’t yet see the direct use case for this.

andrew_presenting_sphinx
For me it was also a first at Engage as I presented a session on Software Documentation. I must admit that I didn’t really relax until after my session was over. If you’re interested the presentation can be downloaded here:

https://www.slideshare.net/AndrewMagerman/engage-2019-software-documentation-is-fun-if-you-have-the-right-tools-introducing-sphinx

One point that was reinforced: if you want to really know something, you should
present it. Turns out the risk of being asked a question that one can’t answer
is a powerful motivator. In the case above, there was a certain amount of
voodoo magic in the Makefile syntax, which I had previously just shrugged off
with a ‘well that works anyway’ and which I now understand.

I was impressed that Peter Straxx, of the Belgian GroupWave company (one o SNoUG’s sponsors last year), actually owned a Lotus car and had showcased it at
the venue entry.

As to the venue, Autoworld is certainly impressive and the cars a wonder to look at. It did sometimes feel too big, where even for the first and last keynotes, when we were 440, the place is so spacious that it looked like we were too few.

I learned a lot, ate too much, drunk too much lovely beer, and caught up with
old friends and acquaintances. Wonderful.

HCL Factory Tour 2 – a look into the pipeline for Domino/Notes

by

The HCL Team, lead by Richard Jefts, was kind enough to invite me to its second Factory Tour, this time in Europe, in the HCL offices in Milan, in Italy. Milan is just a 3.5 hour trip in the train from Zürich, going through the new Gotthard tunnel, so it was an almost leisurely travel, compared to the trip to Boston last year.

richard_Jefts

There were 140 attendees, the majority coming from German-speaking countries.

We were treated to a wonderful evening, discovering the sights of Milano, and being invited to the Grand Hotel to eat delicious Italian food. Most of my table asked for a second serving of risotto!

Here are my take-home points:

Better Builds

HCL is modernising the build processes. The aim is to have consistent builds where all the software gets simultaneously built and released. In particular, we’re seeing simultaneous builds for the clients (i.e. Mac and Windows at the same time) and for the products (Domino, Notes, Sametime). It’s a most welcome development.

The last release of Domino saw some quality issues with the Language packs. Kudos to HCL’s Russ Holden for acknowledging the quality issue and dealing with it. Building language packs was, in the past, done by a completely independent department, almost as an afterthought. It turned out that the internationalisation process was not completely automated and needed some laborious manual adjustments. The upside of this is that now that the issue has been surfaced, the core Domino Team will be doing the internationalisation internally, automatically, which means that the next releases will see the language packs being simultaneously released with the main build. This is excellent news for European customers whose software won’t be second-class citizens anymore.

Better Processes

HCL is intent on having a closer relationship with its customers and its business partners, and we were shown many improvements that made me, as a small business partner, smile with barely concealed excitement.

The processes that are being now replaced were unwieldy and frustrating. These were tailored for large enterprises and not to small, nimble companies, and it’s very rewarding to see how HCL is revamping this:

Better support:

We’ve been treated to exceptionally fast support over the past year coming from HCL. The presence on Twitter is remarkable, and the speed with which issues are cleared is a real relief, and a real surprise compared to the previous, glacial pace. Kudos to Michael Fiorentino for the sterling work here.

Previously, business partners were not able to register issues and bugs directly, this will now change.

Better interaction with the customers:

HCL introduced their customer advocacy program, which will give you as a customer direct access to someone within the development team. This is such a good idea.

If you are a customer, register yourself on:

https://www.cwpcollaboration.com/advocacy.html

Closer relationship with the business partners

Speaking as a small provider, IBM’s process for obtaining licenses was so complicated, slow and heavy that it was not commercially viable to resell licenses, or to bundle services with installations. HCL’s processes are far more direct and the licensing model for new licenses will be far simpler. This opens opportunities for SMB clients (and the smaller, local providers which these favour).

If you are a business partner, be sure to register on:

http://www.hclpartnerconnect.com

Better Software

HCL Nomad (Notes Apps on Tablets)

We were shown a very polished, very quick demo of HCL Nomad (a Notes Client working within a tablet) on an Android device. The delicious side-effect of porting the code to Android is that HCL used OpenGL, which means that the codebase can be compiled as WebGL, which means it can be compiled for browsers! Yay.

There is still some work to be done as to the distribution mechanism, but I would say this is coming close to a shipping date.

Verse on Premise

Verse is also getting a makeover, including a much-expected revamp of the calendar view.

Better clarity on the future of the clients.

I was confused about was how to square backwards compatibility with a smaller, quicker client. As far as I have understood (the following here are my opinions):

The thick client based on Eclipse is going to stay on with minimal improvements so as to ensure backwards compatibility for customers that depend on the full range of the thick client’s capabilities.

HCL Places is still a work in progress and will include some Low-Code options, apparently with an engine based on Node-Red. This seems to imply that HCL Places will try to integrate into cloud-based solutions with microservices.

The new kid on the block (for me) was the upcoming lightweight client. As I’ve understood, this will integrate the HCL Nomad code for interacting with applications, and Verse for the Mail experience. This is reassuring, both components seem well on their way and conceptually seems something which will be relatively straightforward to implement.

Search and DQL – security model will be seamless

The security model of Notes is the single biggest advantage the platform has over all others. As a developer, not having to worry about who has access to what data, once this has been determined on the record/document level, gives me great speed in development.

The demos we saw of DQL and of searching with Elastic Search were using Anonymous access. I was worried that we would lose our edge if the security model was not respected at a low-level, but HCL assured us that they are very aware of this. I’m hoping that the next demonstrations will include authorization and authentication.

Focus on application development.

vowe_in_milano

We had a surprise final keynote delivered by Volker Weber of vowe.net fame. I was delighted to see that Volker’s skills in stripping down things to the essentials was reflected in his presentation’s format – a series of pictures with no text, and the presentation’s content – which was to recognise that the core feature of Domino is cheap, robust app development. I couldn’t agree more, and I’m convinced that HCL agrees wholeheartedly too.
These are exciting times ahead for our community.

Deep dive into HCL’s strategy for Domino/Notes

by

HCL was kind enough to invite me to their CWP factory tour themed from the story Charlie and the Chocolate factory, held in Chelmsford, Massachussetts on July 11-12 2018.

oompa_loompas

We already knew from Engage and DNUG that Richard Jefts has no qualms in donning a silly costume, but seeing Jason Gary as an Oompa-Loompa was truly memorable. We now have photos and a means to blackmail them.

Many thanks to them and the HCL Staff for organising the event, it was definitely worth the trip.

Letting the developers loose.

Over the course of two days, the HCL Staff, the core developers of Domino 10, showed us what they had been working on. We had unprecedented access to the core team, and the interactions – on the first day, a series of presentations/workshops and on the second day, an impromptu ‘Ask the developers’ session –  were very fruitful. And the communication was two-way too – on numerous occasions, when we made requests, the developers were surprised or suddenly understood where we were coming from.

I had a sense of the developers being let loose after years of being held back, and it’s obvious that they are completely aware of the pain points that external developers and business partners have had, and are happy to have the opportunity and resources for actively solving them.

ask_the_developers

don’t ask about the chicken.

HCL’s strategy

The penny dropped a number of times as I suddenly started to understand exactly what was coming our way, and how the different improvements and new developers fit together in the broader HCL strategy.

Business: play to Domino’s strengths: easy AppDev and secure, low-maintenance, on-premises infrastructure

darren_oberst

We had the pleasure of listening to Darren Oberst, who heads HCL’s PCP division and explained to us the logic of HCL’s investment in the IP of Domino/Notes. And it’s consistent. HCL has recognised that not all customers are cloud-frenzied and that having one’s own infrastructure, on-premises, is valuable when data privacy and control is important. This is especially important in the German and Swiss markets.

It makes a refreshing change to IBM’s mantra of ‘cloud first’ which left Domino sticking out as a sore thumb in IBM’s portfolio.

HCL are making the web-based mail client, Verse, work well on-premises, for instance without the need of a separate IBM Connections environment.

There is a sweet spot for Domino as a ‘one-server-does-all‘ for the small and medium businesses which was largely ignored by IBM in the past.

I’m hoping that, with some judicious investments in modernising Domino, we business partners can start offering Domino to completely new customers, instead of restricting ourselves to customers who already have an installation.

Increase robustness of the Domino server

There is a team called ‘Total Cost of Ownership’ whose purpose I didn’t initially understand. The penny dropped as I realised that the work is to increase the robustness of the Domino server, to make it an attractive investment for SMBs also, who don’t have the size to have their own dedicated staff but still want an on-premise infrastructure. In short, to make Domino an ubiquitous offering, which is great!

Out of the many improvements that were announced, the ones on the 64GB database limit was very welcome, as was the commitment to breathing some life into Sametime, including getting rid of the monstrous Websphere database in the background. My jaw dropped as one of the casual comments was that the Search was going to be upgraded so that the indexes get immediately updated with new entries. This has been such a pain point in the past explaining to customers why they can’t find the information they’ve just inputted, and I wasn’t expecting a resolution any time soon.

Impressive, also, were the self-repairing logic that is going to be included. Fixups happening automatically, automatic checks that replicas are correctly distributed, and automated recoveries of corrupt databases. This goes completely in the direction of making the domino server so reliable that it can be offered to SMBs without qualms.

Node.js opens up Domino to modern web developers.

It took me a while to get this. The essential point is that with the new integration with node.js, modern web developers do not need to have any Domino-specific knowledge to start coding with Domino as a back-end NoSQL server. All this new stuff is not aimed at hard-core Domino developers, who already know all the hoops and workarounds needed to get Domino speaking to other systems. It’s ‘code with Domino without opening Domino Designer’.

The DominoDB module will be available as an npm package. Download it, require it, and you can create a connection to a database and start making CRUD operations against it. To complement this (and this was a welcome surprise here) the HCL Team are working on a SQL-Like query language (Domino General Query Facility) which will return JavaScript Arrays of documents. This promises higher query speeds and will be available from any of the languages, and can be run by any of the Domino languages.

Another pain point that was addressed was Identity and Access Management (IAM) which will follow OIDC/OAuth standards, and will be able to be accessed as a service.

Hackathon

Three teams participated in HCL’s Hackathon. (I can’t yet say what one of the teams did)

‘web app start’ application

runners_up_hackathon

Graham Acres, Richard Jefts, Bernd Gewehr, Jason Gary, Christian Güdemann, Thilo Volprich, Thomas Hampel

This team created in no time at all a ‘web app start application’ that locally stores credentials PWA Application that could connect to a Domino Server once and once only and then securely locally stores the credentials and allows to redirect to other domino-based web apps. Wonderful, and a tour de force in the time available.

Enabling Domino Compatibility for modern apps built with Play Framework

hackathon_winners

Richard Jefts, Dan Dumont, Andrew Magerman, Ellen Feaheny, Jason Gary, Steve Nikopoulos, Dave Delay,

Our team pursued a more conceptional idea (we didn’t code anything).

Ellen Feaheny leads AppFusions, a company that sells IAAS (Integration as a service) products, and came up with an idea of trying to leverage Domino with apps built with the Play Framework.

Play, by Lightbend, is used by enterprise corps for micro-services architectures using Javascript, Scala, or Java. It is especially good for apps that require large scaling design from the get go.

We kidnapped the HCL Team doing the node.js integration and brainstormed with them on how this could be implemented, sorting through the high-level requirements.

How does the gRPC protocol work?

Steve Nikopoulos, and Brenton Chasse spent a long time holding my hand and explaining to me slowly how the gRPC protocol works.

As far as I’ve understood:

  •  It’s a very efficient protocol between two machines to deliver remote procedure calls, i.e. calls to execute something on a remote machine, using commands that look local to the caller.
  • The transport is done over HTTP 2.0 (so there’s already a speed advantage because the connection remains open).
  • The messages sent are small, because they are just within the TCP envelope of the HTTP call, and because the format of the message is predetermined and the declarative part of the message is kept to a minimum through diverse tricks.
  • Crucially, messages can contain other messages, so you have an out-of-the-box possibility of sending a tree of nodes, which I found very satisfying.

Think of it of as a very clever lossless compression for messages which have a predetermined format (for instance, ‘bulk update documents’, or ‘delete this document’).

The starting point for defining the protocol is a file called a .proto file which defines very exactly the data structure of the messages. (see grpc.io). Once these are defined, you use a protocol buffer compiler to generate data access classes in any of the supported languages (all the favourites are already there).

Put it another way: gRPC comes with tools that enable you to create ‘stubs’ in any computer language that you want that will be able to ‘compress’ and ‘decompress’ the message. As an analogy, think of the stubs that were generated when building SOAP interfaces.

Our idea:

If HCL makes the .proto file for the gRPC protocol public, then anybody can create an interface to the gRPC Server which will come with Domino.

This was just a concept, and there are still some open questions, i.e. who supports the solution in case something goes wrong, how does one organise versioning, how does one ensure compatibility between different versions of the protocol…

Andrew Davis told us that the HCL Team loved the idea, and that idea was rewarded by the first prize! In fact, Andrew even committed in front of the audience to work with AppFusions (and others who wish to get involved) to see how this could be made real, sooner than later.

Conclusion

Domino is going places. The CWP Factory Tour was an exhilarating meet-up of the core Development team and the core Domino community, with very productive exchanges of ideas. I haven’t been this optimistic about the future of Domino/Notes in years. Watch this space!

Please call me if you want some help or advice about the future of Domino!

cwp_attendeesThe lucky attendees

Modernization of Ortho-Team’s Reservation System, a success story

by

In 2017 Ortho-Team released a new version of their website, and the optics of the reservation system clashed with the new website. Something had to be done.

Ortho-Team is a leading Swiss provider of orthopaedic products. Most products are made to measure for its customer, which means every sale is linked to at least one appintment for advice and measurement. Ortho-Team had a reservation system based on its IBM Domino infrastructure. Customers can choose an appointment for the location and product they need. These appointments are dynamically calculated from the available free time of all Ortho-Team employees.

The original system was built using classic Domino Web technology and was working reliably and robustly. But the design had become old-fashioned, and crucially the reservation system was unusable on mobile phones.

Decision: keep as much of the existing system

After considering several different scenarios, we decided to stick to the classic, robust technology in the back-end and revamp the UI with a modern, reactive interface. The reasoning was that this had the lowest potential impact, and the costs would also be the lowest for the desired effect.

The User Interface Design and coding of the necessary HTML and JavaScript files were done by the talented team at e621.ch. The coding on the Domino side was done by Magerman GmbH.

We’ve shown the results in the following pictures. The transition is now seamless!

If you want to update your existing classic Domino Web applications, don’t hesitate to contact us!

Lessons learned:

  • If you have classic Domino Web applications, it’s possible and cost-effective to revamp the UI using modern, reactive Front-End technologies.
  • The biggest limitation is that we had to stick to the original workflow of consecutive sequential pages. In Classic Domino Web Development, business logic is carried out on the server by serving sequential pages, and we wanted to keep the business logic where it was. A ‘single-page’ UI was not conceivable.
  • We stumbled from time to time on technological limitations of the Domino platform, but could quickly find alternatives.
  • From the start, we set up a testing environment which was accessible to all parties involved. This proved to be very effective; we ended up finalising the end product through many small iterations.

Engage 2018 recap

by

Engage 2018 was the best Engage I’ve ever attended. Not only was the organisation and venue superlative, but the recent energy of the new HCL team was infectious.

IBM’s Bob Schulz delivered a soporific and uninspired Keynote speech, in sharp contradistinction to Richard Jeft’s and Andrew Manby’s energetic, and funny, presentation. Their enthusiasm was infectious.

OGS_Engage

New announcements:

HCL Nomad

Most promising was the presentation by Andrew Davis showcasing Nomad, which is a Notes Client running on an iPad. He demoed the current version, which only works online. They are working on improving the controls (dropdowns, checklists etc.) to work on a touchscreen, and that seemed to be working well. Links (doclinks, database links) are working.

The next necessary step is making Nomad work offline, which would imply implementing the replication engine, but also implementing a local full text indexing. I imagine that that will be challenging. Promising, also, is the extension of LotusScript to be able to access the mobile devices’ goodies such as cameras, location.

I can see a good use case for field service workers who need a fully offline machine, and I can also imagine this would be a very easy way to surface pure Notes Client applications to the iPad with very little cost (since the codebase must not be modified).

I’m not convinced with the attempt to make Notes applications work on the iPhone, the format is just not compatible with the existing application layouts.

Andrew_Harris_Engage

Andrew Davis showing how Node.js will be integrated into the Domino stack – architecturally elegant, methinks.

HCL Places

A surprise was Jason Gary’s prototype demonstration of ‘HCL Places’, an universal integrated chatclient/Notes Client which would run in a single application, with data storage on the Domino server. It’s one of those products that HCL can market independently. Looks promising, but probably needs a lot of work for it to fly.

HCL_Places

Notes Client:

Ram Krishnamurthy showed some of the new changes in the Notes Client. I found most of them small incremental UI changes. The Workspace is getting a makeover, and I hope that what was shown is not the latest offering – the spacing of the tiles was all wrong in my opinion.

The real improvements was the decision to merge both mac and Windows codebases for the client (presumably as a result of high mac usage within IBM) and the simultaneous release of all codebases (including Sametime, which caused much of the headaches of Domino 9.01FP10). Both these changes should improve the quality of the delivery.

As a side note, it’s interesting that both HCL Places and Nomad are integrating the ‘pure’ C++ Notes client. This reinforces my feeling that there is true beauty and robustness in the ‘small’, ‘basic’ client, and that the 2008 revamp using Eclipse RCP is just a flaky, clumsy abomination whose costs (slowness, size, difficulty to configure, architecture breaks) are not compensated by its advantages. HCL announced they would sunset unused functionalities, and that is welcome. Hogne B. Pettersen suggested to ditch the RCP implementation entirely, and I  secretly cheered.

XPages:

some small improvements announced, but I think it’s clear XPages are a dead end.

New interesting technologies and skills:

Of particular interest was Knut Herrmann and Paul Harrison’s session on Progressive Web applications, an easy way to adapt a web application so that it behaves like a native mobile app, without the hassle of passing it through the testing of an app store.

ENGAGE_2018 - 2

Also fascinating was Paul Withers’ and John Jardin’s session ‘Domino and Javascript Development Master Class’. Node.RED is definitely a technology I’d like to look in. As a tip, Paul suggested making REST Tests directly with Node.RED, and not with Postman, for instance. And Node.RED is an easy way to deliver scheduled actions.

ENGAGE_2018 - 1

People

Most enjoyable is that Engage is meeting the community. Probably because of the long history of the platform, real ties of friendship make this community special, much more than any of the other gatherings I’ve attended over the past years. There is always somebody who can help!

IBM_Champtions_Engage

IBM Champions on stage

Can HCL deliver?

That’s the crucial question. Whereas I am impressed by the new team’s vision and obvious energy, my gut feeling is that they are overambitious and risk underdelivering. Jason Gary mentioned ‘we’re just like a startup’, which made me wince since in this space, with a large legacy codebase, you don’t have the luxury of a green field. The risk I see is V10 breaking existing functionality, which would be fatal.

During questions and answers I asked Richard Jefts: ‘who was responsible for the Notes 9.01FP10 release?’, as the quality assurance on that release was sub-par. Richard took it on the chin, to his credit, assumed responsibility, and told us that a separate QA post had been created to avoid this happening again. There again, the risk here is delivering under time pressure a software product that isn’t sufficiently tested.

Thanks to the Engage Team and the sponsors

Theo Heselmans made a sterling job of organising the event, as usual, and the venue was particularly well suited. The venue was the spectacular SS Rotterdam, a permanently moored cruise ship. The sessions were in the ship, we slept and ate in the ship. Many thanks to the sponsors which makes this event possible, and free!

SS_Rotterdam

On a personal note, the gods of luck were with me and I won two the two prizes that tickled my fancy: a LEGO set from OnTime and a Raspberry Pi housed in a BBC Micro B housing from LDC Via. Awesome!

andrew_magerman_ldc

Matt White from LDC Via presenting me with their awesome giveaway – a Raspberry Pi housed in a BBC Micro B

From 3000 milliseconds to 200 to open a page: Speed up your Notes Client applications by discovering their bottlenecks.

by

I had inherited a template for a database which was taking 3 seconds to open any sort of document. It soon got to be irritating, and I noticed that the little lightning bolt at the bottom left side of the Notes client was flashing wildly, so I fired up the Notes RPC Parser to try and see what was all the traffic about.

The culprit was soon found: it was a subform containing buttons whose label was dynamically computed depending on the user’s desired language. And, unfortunately, it was making an @DbLookup for every single button, and an uncached one at that. If you look at the screenshot, you’ll see that an @DbLookup generates 3 NRPC Calls, for a total of 50ms – and this was done 30 times each time a document was opened.

That, and another issue that I corrected, meant that I could massively improve the performance of the application for only about half a day’s worth of work.

screen-shot-2016-11-30-at-16-58-34

Feel free to use the tool, or if you can’t be bothered, hire me to solve your Notes Client performance issues.

P.S. I’ve just released a new version of the tool Notes RPC Parser, which you can download here:

Back from ICON UK with some goodies.

by

Another year gone and ICON UK did not disappoint. Tim Clark made a wonderful job of continuing on the success of last year’s performance, bringing back the event to two full days. IBM graciously hosted us in its stanley kubrik-esque Client Center, with stunning Thames views, lovely food, good infrastructure. Wonderful.

René Winkelmeyer held a wonderful session on gradle and I was very impressed by his mastery of his computer. He was actually using vim to edit text files, and I must admit that I was so impressed that I am actually writing this blog on vim, learning the hard way. Really fast, configurable, powerful text editors that are done for programmers seem to be more and more the norm. Beside vim, there’s sublime text, Matt White mentioned Atom as being his favourite, and I must admit that it is really a refreshing break after Eclipse, which is slow and ponderous in comparison. I’m not even making the comparison to DDE.

Here are the slides: http://www.slideshare.net/muenzpraeger/iconuk-2015-gradle-up

Speaking of gradle, that’s another trend which I can see happening in parallel in several different systems. I’ve been discovering the joys of Linux and scripting because of a small raspberry pi project I am doing on the side, and there is a common theme of self-updating, self-building systems, be it apt-get (for linus os x updates), or bower (for javascript libraries), or homebrew formac os libraries, or maven for building up the dependencies for a java project. The skillset needed to build a modern project is getting to be more and more to know which great big building blocks are needed and mastering the building tool. I’ve been trying to get my head around Maven right now, but since I heard some dark mutterings by Paul Withers about the documentation of Maven, I think I’m going to just jump over Maven and go to gradle directly.

I had the pleasure attending the session by Bill Malchisky and I’m proud to say that I understood at least half of what he said. He speaks surprisingly fast and surprisingly exactly; it’s an uncommon combination but one really needs to listen hard. He is also a script master, and again, eerily, a nudge in the direction of ‘invest in your text-editor and typing skill’.

Matt White showed the magic of node.js, which is used extensively in their solution LDC Via, and there again I was seduced by the simple structure, and the promise of only a single thread working very very fast.

I spent a thoroughly enjoyable hour with Serdar, and I discovered that we share many opinions as ‘convinced skeptics’. It was a pleasure to bash on pseudoscientific nonsense with him. Next time I’ll bring woowar in and we’ll do a bigger skeptics session.

On the second day Andrew Grill showed the advantages of Connections, his style was entertaining and persuasive. His colleague Harriet explained to us rather condescendingly what it was to be a millenial. I didn’t understand why being impatient and having a short attention span is somehow good, and I took exception to the comment that millenials don’t read instruction manuals but just expect things to work immediately out of the box. Surely that is a result of extraordinarily good product development (I am thinking in particular of the Apple products), and it’s not because the millenials are this super-brainy generation. Making things simple is extraordinarily difficult. Just try it.

Engage 2015 as engaging as ever

by

Engage is the most successful LUG in Europe, and as usual I am slightly bewildered by how Theo Heselmans, our gracious host, manages to pull it off. The venue was lovely, the opening session room stunning, and the content was very high quality. I really enjoyed meeting many community members whom I had only seen online, including a couple of my Stackoverflow saviours (Per, nice to have met you!)

The city of Ghent itself was a nice surprise. The inner city is full of history, with many old buildings harking back to a more prosperous past, and a surprising number of churches. I had a little walk to the north of the city, though, and it’s obvious that the city went through an industrial phase which it got out of and has not really recuperated from.

Here are my technical take homes from the whole two day session:

Both Ulrich Krause and Frank van den Linden independently confirmed that they didn’t like the new ‘Java’ element and found the oldskool WEB-INF folder stabler.

Theo Heselmans presented some of the Javascript frameworks he’s been using; I knew Bootstrap and Backbone; he recommended Ratchet and Knockout as well. Also, if you want to store local stuff, the way to go nowadays is no longer cookies but the manifest or local storage.

John Daalsgard had a good session explaining the Domino REST API; I learned lots of stuff but was sort of disappointed that authentication was not really talked about. Most of the examples were using anonymous access, and authentication is still not really an easy thing to do. Paul Harrisson, who did the local web application for engage, pointed me to his blog entry about authentication. I’ve been working with Julian Buss’ framework DominoToGo and I was initially under the impression that the REST Services introduced in 9.0.1 would mitigate its usefulness but I’m coming to the conclusion that as soon as you get out of the demo-cases ‘simple text’ and ‘anonymous access’ things start getting complicated using REST, i.e. one has got to start coding things oneself.

One of the most interesting sessions was the one on GIT done by Martin Jinoch and Jan Krejcarek. Martin was very stern and he endeavoured to persuade us to abandon the idea that the source code resides within the NSF and that the Git repository is the backup. Rather, the source code is in GIT and the NSF is just a throwaway, last-minute build construct. I almost broke in tears. Martin also admonished us to turn everything off that automatically builds, including the nsf to on-disk-project sync.

I was also relieved to hear that other fellow developers were irritated by ‘false positives’, i.e. files that have been touched, and therefore appear in the staging area of git, but whose code has not been practically modified, and therefore are really cluttering. There is a project called DORA which alleviates this, but it only works if one starts the project with it. Implementing it midway is bad, apparently (thanks to Serdar).

The London Developer Co-op was there in force, with a stand even, and showed us a very polished product for data exporting. I can see use cases if customers just want to store their data somewhere else, to finally kill off the remnants of their Domino infrastructure, but the fact that the business logic does not get exported will still represent a large exit barrier.

Mark Leuksink and Frank van den Linden introduced me also independently to bower, a package manager that manages the javascript library dependencies automatically for you. The idea here, if you’re doing an XPages project, to have bower point at your ods structure and do the updates here. You’ll need to press F9 in the package explorer before syncing the project.

In the mindblowing categories, Nathan Freeman showed the Graph construct he has made available within the OpenNTF Domino API. Documents stored in nsf without views? That’s just weird. Possibly illegal. And whereas I can see obvious advantages in terms of speed when the data structure is already known in advance, especially for transversal, multi-layered searches like ‘show me the persons who know the persons I know’, I’m not sure how the Graph concept would deal with ad-hoc requests, or with a change in the underlying data structure. I would really like to see what sort of measurements one can make as to the performance of data writing and reading, especially in large numbers. The demonstrations as well were built from scratch, and worked well, and I’d be very interested to see what happens when one takes an existing data landscape and ‘graphs’ it.

The final session I attended was from Paul Withers and Daniele Vistalli. Paul presented the newest possibilities of the next version of OpenNTF Domino API. They are introducing a concept of metaversalID which is a combination of database replicaID and Document Universal ID, and apparently the code has been made Maven-compatible. It looks like we will have, in conjunction with Christian Güdemann’s work on an eclipse builder, soon a system where we can start thinking of continuous builds. We’ll be big boys, then, finally.

Daniele introduced the Websphere Liberty Server. I had dismissed the Websphere server as a huge, lumbering IBM monster but apparently the Liberty Server is small and lightweight. And then, doing some magic, Paul and Daniels made the Liberty Server behave just like a Domino server. The demonstration was still very much in beta stage, and I’m not clear as to the implications of this tour de force. But it might be a game-changer.

my non-technical take homes:

When travelling, bring two phone chargers. With the iPhones losing juice so quickly, losing your charger leaves you strangely vulnerable and incommunicado. Thanks to Ben Poole for letting me load up at the LDC Via stand.

It is unwise to start debating with Nathan Freeman at 2.30 in the morning after everyone else has been kicked out of the hotel bar, and Nathan has a bottle of tequila in an ice bucket.