getting git

(git is wonderful. In the same way that unix is an’operating system done right’, git is ‘source control done right’.

It has a surprising history, being basically the creation of Linus Torvalds, who decided that all the available source control solutions were useless, and so just wrote his own. That is fabulously cool.

This is a small article to introduce you to git.

Learning Curve

There definitely is a learning curve with git. At least that was the case for me.

For starters, I had to throw away some concepts which I thought were set in stone. Git forces you to think about code in a different way. It turns out that that way of thinking has large advantages.

The single biggest mental stumbling block for me was that I always considered the real code, the precious bit, to be the one I’m currently working on, and the code that is in source control system to be a backup. A byproduct, so to speak. You need to completely overturn this with git.

The real stuff, the precious stuff, is what is in your repository. What you are currently working with, your working directory, that’s something like a scrapbook. You’re trying things out, making changes here and there, but it’s all very temporary and it’s not where the real stuff is. It shouldn’t be frightening to overwrite your working directory, for instance.

Repository

Let me track back a while and introduce the three areas in a local git repository (I am not talking about a remote repository, I am talking about a local git repository, on your local hard drive, and blazingly fast).

The repository is where the history of your project is stored. It’s quite similar to a series of snapshots of your whole project. Each of these snapshots contain your whole project (technically, git is not saving the whole project but just saving the delta to your initial commit, but conceptually, think of these snapshots as whole copies of your project).

These snapshots are called commits. They are unique, and can be referred to in a number of different ways, either directly (by using its number, which is a SHA-1 hash) or relatively (three commits before the last one I did).

Each commit points to a parent commit. When you have two different commits pointing to the same parent, these two commits are the starting points of different branches. The two codestreams, from this branching point, starts having a life of their own, although it’s possible to merge different branches together at a later stage.

Decentralized weirdness

This is a good time to introduce another weirdness that took me some time to grok. There is no primacy between branches. It’s a delightfully anarchic system, no bosses – a little like the anarcho-syndicalist communes of ‘The Holy Grail’. There is no master control program, no centralized master system. One can define certain branches to be more ‘important’ than others, but it’s just a naming convention, it’s not something that is built in.

The same applies between your local repository and the remote repository. Git really doesn’t assume that one of them is more important than the other. It’s very disquieting, but liberating once you’ve understood it.

Working Directory

Next, your working directory. This is where your whole project is, where the code is, images, what have you. This is what gets ‘snapshotted’ each time one makes a commit. It could be possible to snapshot your whole project every time, i.e. every single file, but it turns out that that is a bad idea, because you end up saving too much and it gets really difficult to understand what exactly happened between two commits. I must admit to having done that at the beginning, because I was blocked in my ‘backup’ concept.

No, what you want to do is make lots of small commits, with each commit changing as few files as possible, as long as they pertain to one common change.

Staging Area

Enter the staging area. Really well named, this is where you determine which files you want to belong to the next commit. You’re not interested in the files that have changed only because something trivial like a timestamp has changed, and you’re not interested in unrelated changes either. You really want to encapsulate all the changes necessary for a single debugging or a single new feature into a commit.

Overview

The following diagram shows the basic logic: you add files to the staging area with the git add command, then you commit those added files with a git commit command. Bringing back a specific commit to the working directory is done with the git checkout command.

sublime_text_graphviz_C8OsfB

So, you’ve got this conceptually? Working directory, repository, staging area? Right. This is what you should work with.

I got very confused by how this is actually built. The repository and the staging area are stored in a folder called .git which is contained within the working directory.

This made my head wobble. Surely, not inside the directory? Surely, that way lies folly? Infinite recursions? Surely it should be external? But that’s how it’s built. Inside. It’s elegant once you start thinking about it but it made my head hurt.

Start with the command line

I would recommend staring with a command-line interface. I know that there are many GUI Interfaces out there (SourceTree being a particularly good one), but all they are doing is adding an interface on top of the command-line instructions.

I would recommend moving to a GUI once you’ve learnt the basics of git. Additionally, the command line is very helpful for noobs, as it notices when you have typed something that doesn’t make sense and makes a suggestion. Stackoverflow and the git documentation are the place to go if you get stuck.

http://gitref.org/ is a good, compact resource.

You can download the git command-line bash here.

Recommended Reading

This is a tip from Eric McCormick.

gitinpractise

Git in Practice [With eBook]

Useful Commands

Here I have listed those commands which I am using regularly and feel comfortable with:

Initializing

git init

First initialization, creates the magical hidden folder .git. This is where the actual repository is stored (see above). You’ll need to select the directory which you want to ‘track’

Staging

Staging means ‘saying these are the files I wish to commit’

The command for staging is

git add <filename>

alternatively you can use

git add .

which will add all the files in the current directory (. is the current directory)

Committing

git commit --message='Committing message'

The convention is to write a message with multiple lines, a bit structured like an e-mail. The first line should be like the subject of the e-mail, the other lines are the ‘body’ of the e-mail.

Creating a remote repository:

git remote

shows the remote repository linked to the current repository. You’ll need to have created this beforehand on the github site.

I use github, so the example here is with github. First log into GitHub and create a repository there. Note the URL of the remote repository (there is a convenient button for this)

git remote add origin <your remote repository URL>;

Note: origin here is one of these conventions, it’s the default name of the remote repository. Don’t make the mistake of naming one of your remote branches origin. Remember, convention!

git remote -v

shows the urls linked to your remote repository.

git remote show origin

This command shows all the branches on the origin repository (the remote one, origin is its name by default)

 

 

Fetching

git fetch

fetches the data from the external repository but does not merge the data.

Pulling

git pull

fetches, then merges the data.

Pushing

git push <remote> <local branch name>:<remote branch to push into>

pushes the data from the local repository to the remote one.

Checkout (restore files or change branch)

git checkout

This switches branches or restores working tree files. It’s a bit of an uncomfortable command at first because it seems to be doing two completely different things. Actually, it’s doing the exact same thing – it’s overwriting your working directory with information from a particular commit. Your working directory is a scrapbook, remember?

I find this useful to avoid committing fluff (my technical term for files which have, technically, changed (a timestamp for instance) but which are not actually changes.) I add the files that I want to have committed with gid add, then I do a git commit, and then a

git checkout -- .

to overwrite the fluff in the working directory.

If you notice you’ve just committed nonsense (in this example I replaced the whole About This Database with nothing), you want to restore the file to what it was before. Here comes git checkout for helping again:

git checkout HEAD~1 -- odp/Resources/AboutDocument

HEAD~1 is a way to refer to a specific commit, i.e. ‘The commit on HEAD’ minus (~) one (1)

Conventions for naming files and directories:

dash dash

If you see before a filename, it’s a disambiguator saying ‘what follows are necessarily files’

dot

. is the current directory – so, everything.

Syntax for defining an entire directory with everything in it

odp/Resources/ would define that particular directory, along with anything inside it. Note the trailing slash!

Conventions for naming commits:

Naming conventions:

HEAD: Head is a pointer the commit that was pushed into the working directory last. Think of it as ‘commit corresponding to the current state of my working directory before I made any changes’.

<commit name>~1 The tilde here means ‘minus’, so this would be the previous commit.

Configuration options that are useful

In git Bash, I immediately increase the font size to something readable with our big screens, and then

git config --global core.autocrlf false

which avoids the annoying ‘lf/crlf’ comments, since I’m mostly working in a windows environment (sniff).
Do this before your initial commit! (see https://help.github.com/articles/dealing-with-line-endings/)

Setting up SSH keys with Git Bash

Since I’m lazy and don’t want to type in password and username every time I connect to git, here’s how to do it it cleanly via ssh.

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

Press Enter for default in which to save key enter a passphrase (a complicated one, alright?)

Add the keys to the ssh-agent (This is a background agent whose job is to always automatically enter your passphrase in. You should be aware that this makes your local physical machine a lot more vulnerable)

 eval $(ssh-agent -s)
 ssh-add ~/.ssh/id_rsa

copy the PUBLIC key to the clipboard (windows)

clip < ~/.ssh/id_rsa.pub

or mac OS

pbcopy < ~/.ssh/id_rsa.pub

add the ssh key to your github account Test the keys with

ssh -T git@github.com

Ignoring certain files or directories

There are some occasions when you don’t want certain files to be tracked by git. Typically, there are for instance binaries of your source code, or hidden files generated by your ide.

You can see which files are not being tracked with this command:

git ls-files --others --exclude-standard

or alternatively,

git add -A -n

There are unfortunately many different ways to define the excluded files. There is a central configuration file, whose location one can find with

git config --get core.excludesfile

Within your project, there are two files that control the exclusions:

.gitignore
.git/info/exclude

Confused? Here is an explanation from Junio Hamano (the maintainer of Git)

The .gitignore and .git/info/exclude are the two UIs to invoke the same mechanism. In-tree .gitignore are to be shared among project members (i.e. everybody working on the project should consider the paths that match the ignore pattern in there as cruft). On the other hand, .git/info/exclude is meant for personal ignore patterns (i.e. you, while working on the project, consider them as cruft).

This command combination is practical (thanks Akira Yamammoto)

git rm -r --cached .

(recursively remove all the cached files (i.e. tracked) from the current directory) then

git add .
git commit -m "fixed untracked files"

Notes/Domino best practices

Best practice when gitting an odp project: Never disconnect the nsf from the eclipse project.
Never change the nsf the odp is connected to.

Thanks to Adrian Osterwalder for the tips.

Back from ICON UK with some goodies.

Another year gone and ICON UK did not disappoint. Tim Clark made a wonderful job of continuing on the success of last year’s performance, bringing back the event to two full days. IBM graciously hosted us in its stanley kubrik-esque Client Center, with stunning Thames views, lovely food, good infrastructure. Wonderful.

René Winkelmeyer held a wonderful session on gradle and I was very impressed by his mastery of his computer. He was actually using vim to edit text files, and I must admit that I was so impressed that I am actually writing this blog on vim, learning the hard way. Really fast, configurable, powerful text editors that are done for programmers seem to be more and more the norm. Beside vim, there’s sublime text, Matt White mentioned Atom as being his favourite, and I must admit that it is really a refreshing break after Eclipse, which is slow and ponderous in comparison. I’m not even making the comparison to DDE.

Here are the slides: http://www.slideshare.net/muenzpraeger/iconuk-2015-gradle-up

Speaking of gradle, that’s another trend which I can see happening in parallel in several different systems. I’ve been discovering the joys of Linux and scripting because of a small raspberry pi project I am doing on the side, and there is a common theme of self-updating, self-building systems, be it apt-get (for linus os x updates), or bower (for javascript libraries), or homebrew formac os libraries, or maven for building up the dependencies for a java project. The skillset needed to build a modern project is getting to be more and more to know which great big building blocks are needed and mastering the building tool. I’ve been trying to get my head around Maven right now, but since I heard some dark mutterings by Paul Withers about the documentation of Maven, I think I’m going to just jump over Maven and go to gradle directly.

I had the pleasure attending the session by Bill Malchisky and I’m proud to say that I understood at least half of what he said. He speaks surprisingly fast and surprisingly exactly; it’s an uncommon combination but one really needs to listen hard. He is also a script master, and again, eerily, a nudge in the direction of ‘invest in your text-editor and typing skill’.

Matt White showed the magic of node.js, which is used extensively in their solution LDC Via, and there again I was seduced by the simple structure, and the promise of only a single thread working very very fast.

I spent a thoroughly enjoyable hour with Serdar, and I discovered that we share many opinions as ‘convinced skeptics’. It was a pleasure to bash on pseudoscientific nonsense with him. Next time I’ll bring woowar in and we’ll do a bigger skeptics session.

On the second day Andrew Grill showed the advantages of Connections, his style was entertaining and persuasive. His colleague Harriet explained to us rather condescendingly what it was to be a millenial. I didn’t understand why being impatient and having a short attention span is somehow good, and I took exception to the comment that millenials don’t read instruction manuals but just expect things to work immediately out of the box. Surely that is a result of extraordinarily good product development (I am thinking in particular of the Apple products), and it’s not because the millenials are this super-brainy generation. Making things simple is extraordinarily difficult. Just try it.

Engage 2015 as engaging as ever

Engage is the most successful LUG in Europe, and as usual I am slightly bewildered by how Theo Heselmans, our gracious host, manages to pull it off. The venue was lovely, the opening session room stunning, and the content was very high quality. I really enjoyed meeting many community members whom I had only seen online, including a couple of my Stackoverflow saviours (Per, nice to have met you!)

The city of Ghent itself was a nice surprise. The inner city is full of history, with many old buildings harking back to a more prosperous past, and a surprising number of churches. I had a little walk to the north of the city, though, and it’s obvious that the city went through an industrial phase which it got out of and has not really recuperated from.

Here are my technical take homes from the whole two day session:

Both Ulrich Krause and Frank van den Linden independently confirmed that they didn’t like the new ‘Java’ element and found the oldskool WEB-INF folder stabler.

Theo Heselmans presented some of the Javascript frameworks he’s been using; I knew Bootstrap and Backbone; he recommended Ratchet and Knockout as well. Also, if you want to store local stuff, the way to go nowadays is no longer cookies but the manifest or local storage.

John Daalsgard had a good session explaining the Domino REST API; I learned lots of stuff but was sort of disappointed that authentication was not really talked about. Most of the examples were using anonymous access, and authentication is still not really an easy thing to do. Paul Harrisson, who did the local web application for engage, pointed me to his blog entry about authentication. I’ve been working with Julian Buss’ framework DominoToGo and I was initially under the impression that the REST Services introduced in 9.0.1 would mitigate its usefulness but I’m coming to the conclusion that as soon as you get out of the demo-cases ‘simple text’ and ‘anonymous access’ things start getting complicated using REST, i.e. one has got to start coding things oneself.

One of the most interesting sessions was the one on GIT done by Martin Jinoch and Jan Krejcarek. Martin was very stern and he endeavoured to persuade us to abandon the idea that the source code resides within the NSF and that the Git repository is the backup. Rather, the source code is in GIT and the NSF is just a throwaway, last-minute build construct. I almost broke in tears. Martin also admonished us to turn everything off that automatically builds, including the nsf to on-disk-project sync.

I was also relieved to hear that other fellow developers were irritated by ‘false positives’, i.e. files that have been touched, and therefore appear in the staging area of git, but whose code has not been practically modified, and therefore are really cluttering. There is a project called DORA which alleviates this, but it only works if one starts the project with it. Implementing it midway is bad, apparently (thanks to Serdar).

The London Developer Co-op was there in force, with a stand even, and showed us a very polished product for data exporting. I can see use cases if customers just want to store their data somewhere else, to finally kill off the remnants of their Domino infrastructure, but the fact that the business logic does not get exported will still represent a large exit barrier.

Mark Leuksink and Frank van den Linden introduced me also independently to bower, a package manager that manages the javascript library dependencies automatically for you. The idea here, if you’re doing an XPages project, to have bower point at your ods structure and do the updates here. You’ll need to press F9 in the package explorer before syncing the project.

In the mindblowing categories, Nathan Freeman showed the Graph construct he has made available within the OpenNTF Domino API. Documents stored in nsf without views? That’s just weird. Possibly illegal. And whereas I can see obvious advantages in terms of speed when the data structure is already known in advance, especially for transversal, multi-layered searches like ‘show me the persons who know the persons I know’, I’m not sure how the Graph concept would deal with ad-hoc requests, or with a change in the underlying data structure. I would really like to see what sort of measurements one can make as to the performance of data writing and reading, especially in large numbers. The demonstrations as well were built from scratch, and worked well, and I’d be very interested to see what happens when one takes an existing data landscape and ‘graphs’ it.

The final session I attended was from Paul Withers and Daniele Vistalli. Paul presented the newest possibilities of the next version of OpenNTF Domino API. They are introducing a concept of metaversalID which is a combination of database replicaID and Document Universal ID, and apparently the code has been made Maven-compatible. It looks like we will have, in conjunction with Christian Güdemann’s work on an eclipse builder, soon a system where we can start thinking of continuous builds. We’ll be big boys, then, finally.

Daniele introduced the Websphere Liberty Server. I had dismissed the Websphere server as a huge, lumbering IBM monster but apparently the Liberty Server is small and lightweight. And then, doing some magic, Paul and Daniels made the Liberty Server behave just like a Domino server. The demonstration was still very much in beta stage, and I’m not clear as to the implications of this tour de force. But it might be a game-changer.

my non-technical take homes:

When travelling, bring two phone chargers. With the iPhones losing juice so quickly, losing your charger leaves you strangely vulnerable and incommunicado. Thanks to Ben Poole for letting me load up at the LDC Via stand.

It is unwise to start debating with Nathan Freeman at 2.30 in the morning after everyone else has been kicked out of the hotel bar, and Nathan has a bottle of tequila in an ice bucket.

Loop elegantly through a JavaScript Array

I’ve been reading JavaScript Enlightenment to try and understand the language. There is beauty, and power, hiding behind the covers of JavaScript, but I haven’t clicked yet. I still don’t really get prototypal inheritance and what ‘this’ really means. There is power in understanding closure and scope, too, I am sure. This small book is recommended to see where the beauty is.

20943392

I’d like to share one bit of code which I found elegant: looping through an array:

var myArray = ['blue','green','orange','red'];

var counter = myArray.length;

while (counter--) {
    console.log(myArray[counter]);
}

It relies on the fact that integer values that are not 0 or -0 are ‘truthy’.

Visually document the dependency tree of Script Libraries in your database with one click

The two ugly sisters of software quality are documentation and testing. Both are sure-fire indicators of how easy a piece of software is going to be to maintain. They are not sexy, though, and most customers or end users don’t care about them.

Nor pay for them.

Bad documentation and missing testing routines makes their hit on quality insidiously, as codebases become incrementally more complex, more difficult to decipher, and changes to the software become, apparently unexplainably, more and more costly, and risky, until it reaches the ‘legacy’ status – change me at your own risk and peril, bwoa-ha-ha-ha!

Manually generated documentation is useless. It’s almost immediately out of date. The only real documentation is the code itself, and that’s why any kind of documentation should be automatic, taken from the actual application. JavaDoc is superb, as was Mikkel Heisterberg’s lsdoc.org (which is sadly not online anymore – please join me in bribing Mikkel to reinstate it).

The Domino Designer Plug-in Dependency Documenter is a modest contribution to help you document IBM Notes applications. Once the plug-in is installed, a click of the button will make a diagram showing how the database’s script libraries depend on each other.

The practical application is to help fight the compilation problems that sometimes happens when Notes gets confused about time-stamps (Generic LSE Failure (no information), anybody?), and a sure way to work this around is to re-save the script libraries in reverse order of dependency. The diagram helps you to find out where to start.

Final Result

Please note that the heavy lifting of the diagram generation is done by Graphviz, which you’ll need to install separately on your development machine. Graphviz is a free, open source library which does a remarkably good job of making clear diagrams based on a simple inputted text file.

The plugin itself is still somewhat rough around the edges, and I’d welcome any kind of feedback or any feature requests you might find useful.

All of this would not have been possible without the previous work of Karsten Lehmann (mindoo), René Winkelmeyer (midpoints), Ulrich Krause (BCC Consulting) and Ralf Petters (Artweger GmbH), all of whom contributed fantastic presentations and blog entries on creating Notes Plug-ins. A special thanks to Ralf, who held my hand whilst I was trying to set up the development environment for plug-in development.

Tim Tripcony’s incinerate function elegantly batch recycles Domino Objects in Java

I recently was surfing through Stackoverflow and I hit a response from Tim Tripcony on this post: What is the best way to recycle Domino Objects in Java Beans

He published a small helper function to recycle an arbitrary number of Domino objects. It’s so simple, and so well written, that I thought it deserved a post of its own. Here it is in all its glory:

private void incinerate(Object... dominoObjects) {
    for (Object dominoObject : dominoObjects) {
        if (null != dominoObject) {
            if (dominoObject instanceof Base) {
                try {
                    ((Base)dominoObject).recycle();
                } catch (NotesException recycleSucks) {
                    // optionally log exception
                }
            }
        }
    }
}

And it’s used like this:

   Database wDb = null;
    View wView = null;
    Document wDoc = null;
    try {
// your code here
    } catch (NotesException ne) {
 // do error handling
    } finally {
        incinerate(wDoc, wView, wDb);
}

I find the function elegant in many ways; the function and the variables are clearly named, Tim uses variable-length argument lists (thats the Object… part with the ellipsis), the new for-each iteration, plus there is some humour. Thanks, Tim.

SSJS is an abomination born out of IBM’s misguided condescension

Scott Souder was recently extolling the skill of the IBM developers and added as an aside “these are not your average domino developers”. I flinched at the statement and thought defensively “does he mean me?”

The more I think of it, though, and it becomes obvious that this is a central assumption that affected the design of XPages. It’s even there, in the introduction to the mastering XPages book.

I would like to humbly submit to IBM that domino developers nowadays are not an unskilled community, and that its modeling of who domino developers are needs some updating, in the hope that this will positively affect further design decisions. We’re not the Samanthas of the programming world.

It’s immediately obvious at any notes gathering that we are not the youngest community – I rarely see anybody in their twenties, for instance. It’s therefore obvious that we are all experienced. Even those without a computer science background (and this applies to me) has learnt, the hard way, that it’s the maintenance and debugging of code which is complex, not the writing. I learned, the hard way, that copy-pasting code is a shortcut which kills you a couple of years later. I learned that structuring code cleanly, with small functions doing a single thing, ends up being far easier to maintain than two thousand lines of procedural code. Making the switch from procedural code to object oriented code helped tremendously, along with the concept of separating business logic from the displaying logic.

XPages was confusing to me at the beginning. It seemed like a step backwards. Squirreling bits of JavaScript into the XML code, into what clearly should strictly be a presentation layer, mixing business logic with presentation logic, after spending years moving code from buttons and hotspots in forms into script libraries? It doesn’t make sense – until you understand the central assumption of IBM, which was that we would not get Java. Too difficult. The dumbo domino developers will never get it. They only get quick-and-dirty RAD.

Then, in the course of my XPages learning curve I stumbled into managed beans, and everything clicked into place. This is where the beauty lies, this is where I can put my layered, easily debuggable, easily testable code. It’s easy to bind to the XML source, and the Expression Language is concise and powerful. And yet these sections are under-documented, the know-how gleaned from stack overflow and numerous blog entries on the subject made by us, dummy domino developers.

In the face of the obvious scarcity of resources that IBM has devoted to the Notes platform, it is such a crying shame to have invested so much into SSJS and the whole formulation of formula language as JavaScript functions. (An impressive achievement, but a misguided one in my opinion).

The resources could have been better spent elsewhere. Learning java is but one of the challenges of learning XPages, and it’s a challenge for which there are tons of documentation.

There’s often a picture made about misguided security in IT – a brand-new lock on a rotten door doesn’t increase your security. Here there is a parallel with misguided condescension – Removing Java out of the learning curve is just a small contribution to making XPages simpler. There’s a plethora of new stuff that we must learn, we must get our heads around conflicting terminology (e.g. Notes View, JSF View, Eclipse View), a lot of functionality comes from knowing that the CSS class you need is called ‘lotusInlinelist lotusUtility’, there are missing tools such as JUnit, refactoring, renaming so you end up having to do a lot of things manually…

SSJS annoys me because of the underlying assumption – I am dumb.

SSJS also annoys me architecturally. It breaks the MVC model pretty badly. It probably also affects performance and scalability (although I have no figures to support this)

And it’s also a misnomer: It’s not at all JavaScript being run on the server (like node.js would do, blindingly fast on top of Google’s V8 engine). At runtime it’s really running as Java. Guess why it takes so long to build your projects? Our poor computers have to interpret the ‘JavaScript’ and translate this into Java code. And your guess is as good as mine when it comes to the new versions of JavaScript – will IBM update too, to ECMA 5 Strict, say?

My tip: circumvent SSJS as soon as you can and go straight to Java.