Thanks John. That is impressive!!
Meanwhile, I have come across the [haskell-diagrams](http://projects.haskell.org/diagrams/) package which is getting closer to what I have in mind. There is also [introductory video](https://vimeo.com/84104226)
During my exuberant youth I also used the xy-pic latex package. I seem to remember that as being more high level than tikz looks to me now, particularly the graph feature, but looking at the xy-pic manual I realize that I remember none of it and the syntax is more cryptic than it seemed back then.
I quite like graphviz/dot and in the past I have used Scheme code to generate dot files for more complex graphs. Unfortunately graphviz does not naturally support some aspects of Penrose notation, like nested nodes for covariant derivatives or having edges connect to specific ports on nodes since where edges attach to a node matters in Penrose notation. These are problems for many graph tools.
This is another reason general drawing tool like haskell-diagrams may be more appropriate.