Ikiwiki users who want to write org-mode files have long only had one plugin for ikiwiki to convert their files to html. That plugin is good as it goes, but it is somewhat limited. Thus, I am happy to announce a new plugin for converting org-mode. It is located in my github repository.

It is still a bit rough around the edges, but it should be completely usable very soon.

File this one in the category of crazy ideas had while running. I was thinking about my new org-mode plugin that I announced yesterday and how hard it is to write in Emacs Lisp. I was wishing that I could write it in Clojure instead. Now, they're not such different languages – they both are evolutions of McCarthy's original lisp, but Clojure is a much prettier, less crufty language.

Let's look at just one construct in both languages: let. In both languages, this assigns values to variable names locally. In elisp, it looks like:

(let* ((foo bar)
       (baz foo)))

Instead of calling let there, I had to call let*, so that the results of the first assignment were known to the second assignment. There is also a lot of parentheses there. (Unparenthesized variables become automatically initialized to nil I believe. I don't think that's a great idea.) Contrast this with Clojure's let.

(let [foo bar
      baz foo])

To me, that's much more readable.

Clojure's core libraries are also much more readable than most of elisp. It seems to me that elisp was designed in the days where computer usage was charged by the letter and has not been updated since. (Don't get me wrong – I love Emacs and would rather program in elisp than Perl, but it has some warts.)

Emacs already has a reader that can be used to turn lispy-looking things into lispy data structures. Add some macros on top of that and you might be able to produce a def that's equivalent to Clojure's. Since much of Clojure is written in Clojure (and all of Clojurescript is), defining the Clojure functions might just be a hop, skip, and jump away from that.

Clojure could intern symbols into emacs (with namespaces prepended) and then emacs could use them. Clojure could also call emacs functions directly – any symbol that isn't defined in a Clojure namespace would be tested to see if it exists in emacs.

One of Clojure's big selling points is its concurrency semantics. I don't follow emacs development very closely, but I don't think emacs is going to support any of that any time soon. Also, would it be possible to compile the new language to elisp bytecode? How would recur work?

There are a lot of questions, and here I am working on a plugin for a combination of tools that hardly anyone uses. But everyone has to scratch their own itch as they say.

Update: As I suspected, this idea won't work: elisp doesn't have reader macros, so the difference in read syntaxes of clojure and elisp would be impossible to express to the elisp reader. Maybe one could write a parser in elisp, but that would probably be slow and hard to do. The idea was only attractive when large parts were already done.

Progress on the plugin is proceeding apace. In fact, the link in the last sentence is a major milestone – for the first time, I can use what ikiwiki calls wikilinks. What would happen previously is that ikiwiki would stick some raw HTML into an org file when it thought it saw a wikilink. Org wouldn't know what to do with this, and would escape the HTML so that it would show up in the browser. This is the reason that I couldn't include images in the posts – they would get interpreted as wikilinks and then org would choke on the resulting HTML.

I'm not sure that inserting local picture files works yet, but inserting pictures from other servers works okay. For example, here is a picture of me standing at the US/Mexico border: and here is me after hiking to the US/Canada border:

I think I have ironed out most of the kinks that caused things to hang previously when updating the site, but more testing is certainly needed. Things that remain on my todo list include figuring out why it doesn't work when the emacs doing the compiling is daemonized, and testing whether locally hosted pictures work.

Any feedback would be appreciated!

I don't mean to be unkind here. xml-rpc.el seems pretty good if you are doing exactly one kind of thing: calling a method on a server that can be reached by http. For anything else, it is very hard to use. My plugin has the following requirements:

• It must be called by another program through xml-rpc.
• It must read from and write to files (and not http).

Neither of these things is made easy with xml-rpc. The first I can understand – xml-rpc would have to somehow insinuate itself into the emacs event loop and watch for calls all the time. The second is less easy to understand. Parsing xml-rpc is not really related to reading from http. So why are the two things tied together? In my opinion, xml-rpc.el would be a much nicer library if parsing the xml-rpc was separated completely from the http stuff. There could be convenience functions, but they would be a layer of abstraction on top of the other two layers.

There isn't a function to get a list of all nodes with a given name? Are you kidding me? Here is my implementation, but I bet there is a better one somewhere:

(defun xml-find-nodes-matching (node name)
  "Returns all children of `node' that have an `xml-node-name' equal to `name'."
  (if (or (eq node '()) (not (listp node)))
      '()
    (if (equal (xml-node-name node) name)
        (cons node (delq nil (mapcar (lambda (nd) (xml-find-nodes-matching nd name)) (xml-node-children node))))
      (delq nil (apply 'append (mapcar (lambda (nd) (xml-find-nodes-matching nd name)) (xml-node-children node)))))))

In general, both xml-rpc.el and xml.el use a distressing number of calls to car, cdr, and cdaddr (and all the versions in between).

Ikiwiki sends named parameters as an even-lengthed xml-rpc array rather than as an xml-rpc struct. This is because not all of the functions that can be called via xml-rpc take named parameters and I guess Perl isn't smart enough to tell a hash from an even-lengthed array. This isn't a huge problem, but it does mean that I need to convert the input to each of the functions that I write into a hash before I use it.

I would like to be able to ignore files that don't have a particular extension. For htmlify, this is the way it works by default. It seems like most of the functions that plugins can implement are not this way by default, though, and that is a shame. If the plugin is not external – that is, it is written in Perl – there is really no problem. The function is called, checks the extension of the source file, and returns without doing anything. However, when the plugin is external and the call must happen through xml-rpc, ikiwiki must transmit the data via xml-rpc and receive the returned data back via xml-rpc. Unnecessary calls take a lot longer in that context.

So I would like for most calls to hook to take an optional extension parameter that takes an extension (or even better, a regexp), and only call the function if the file name has the same extension (or matches the regexp).

Does a function that you're trying to write in emacs lisp give you a string? It's pretty easy to throw it in a temporary buffer and then tell emacs to do the things that you would normally do while you were editing in order to get the proper information out of the string. The with-temporary-buffer macro makes it especially easy to do just that.

One of the things that took me the longest was to figure out what this couple of lines of code was doing:

(setq valtype (car (caddar xml-list))
      valvalue (caddr (caddar xml-list)))  

What should the value of xml-list look like in order to get the correct thing out of it? It turned out that I needed to take the cdr of the cdaddr of the caddar of the parsed xml in order to get the correct value. That only worked when ikiwiki was responding to a method call, though. I had a much easier time getting the right values out when I simply started using the xml-find-nodes-matching function that I showed above.

When you see yourself writing more than a few car or cdr calls in a row (or first, rest, or nth calls in Clojure), stop and try to write a function that finds what you are looking for. The function doesn't have to be recursive, but that might be the easiest way to do it.

In a sense, this is just saying that code can be easier to debug if you're tracing it. But since the calls between ikiwiki and the plugin must go through files anyway, we get the program traced automatically.

Lately, I've been playing around with anything.el, which has been described as "Quicksilver for emacs". I've played around with it before, but have had a hard time changing my habits so that it's the first thing I think of doing when changing buffers or finding files. The main reason, I think, is that iswitch is usually just as fast.

However, I have started using git annex to store my music and videos, and that has added some complexity to the way that I deal with them. My usual flow is to do a git annex find to see what files I have on my laptop, after which I play one of them with smplayer. When I am done with the file, I either use git annex drop or git annex move to get rid of the file from my laptop.

All of that takes a lot of typing in the shell, so I wondered whether I could speed it up using anything. If you guessed that I could (given that I am writing this article), give yourself a gold star.

anything is highly extensible. Anyone can define their own setup with a command like this:

(defun my-anything ()
  (interactive)
  (anything-other-buffer
   '(anything-c-source-buffers+
     anything-c-source-recentf
     anything-c-source-files-in-current-dir+
     anything-c-source-git-annex
     anything-c-source-org-headline
     anything-c-source-bookmarks
     anything-c-source-bookmark-set
     anything-c-source-emms-files)
    "*anything*"))

which is just a function that tells anything to use a bunch of different "sources" as possible interesting things that you might want to interact with. For example, anything-c-source-recentf gives a list of files that you recently had open. That comes with the anything package, though, so it's not too interesting from our point of view. What is interesting is anything-c-source-git-annex. It looks like this:

(defvar anything-c-source-git-annex
  '((name . "Git Annex")
    (candidates . (lambda ()
                    (with-current-buffer anything-current-buffer
                     (with-temp-buffer
                       (call-process "git" nil t nil "annex" "find")
                       (goto-char (point-min))
                       (split-string (buffer-string) "\n")))))
    (volatile)
    (action . (("Watch" . anything-c-git-annex-play)
               ("Drop" . anything-c-git-annex-drop)
               ("Move" . anything-c-git-annex-move)
               ("Copy" . anything-c-git-annex-copy)))))

It should hopefully be fairly easy to understand. There are two interesting parts: getting candidate files, and the actions on the candidate files. Getting the candidate files is as simple as putting the results of git annex find into a temporary buffer (which must be in the same directory as the current buffer) and then putting the lines of the buffer into a list. (This wouldn't work if the file names contained newlines, but I don't have any of those. If you wanted to be really careful, you could add the "–print0" option to call-process and split the string on "\0".)

The actions are all functions. The simplest is anything-c-git-annex-drop, which just looks like

(defun anything-c-git-annex-drop (candidate)
  (anything-git-annex-command "drop" (concat "\"" candidate "\"")))

where anything-git-annex-command just starts a shell command. More interesting is anything-c-git-annex-copy, which is defined as follows

(defun anything-c-git-annex-copy (candidate)
  (let* ((to (anything-comp-read
              "To Git Annex: "
              (anything-c-git-remotes)
              :must-match nil)))
    (anything-git-annex-command "copy" (concat "\"" candidate "\"")
                                "--to" (concat "\"" to "\""))))

(defun anything-c-git-remotes ()
  (with-temp-buffer
    (call-process "git" nil (current-buffer) nil "remote")
    (delete "" (split-string (buffer-string) "\n"))))

The anything-c-git-annex-copy command needs to know to which repository it should copy the candidate that has been selected. I find out which one I want to copy to by listing the remotes that are known about by the git repository that the file is in. Using anything-comp-read means that the remotes are listed in the same style all of the anything candidates are listed in. It's pretty nice, because it keeps me in the same mode of operation as before.

It's a bit hard to write about what makes anything.el an interesting way of doing things in emacs – a screencast might be more helpful – but I hope some of the power is evident from this little writeup. I'm going to try to train myself to use anything more often. We'll see how it goes.

Now I am able to write wikilinks to images on the local server and not just hosted remotely. For example, is an image that I wanted to show at the end of the article on my time-series framework, but which I couldn't because the org plugin wasn't able to show images at the time.

Another bit of progress is that I finally debugged the little shell shim to allow it to start up the emacs daemon properly. Now I can just have one emacs window running and one emacs daemon in the background.

The remaining work left to do should not be too difficult. I have some of my own org customization baked into the file which would be better left up to the user's preferences. I have to decide whether it makes more sense for the user to customize these preferences in the ikiwiki configuration file or just in their .emacs. The latter would be easier for me, but it might be easier for the user (especially if they have multiple wikis set up) to do it in the ikiwiki configuration file.

I also need to allow for verbatim inclusion of links. I guess it makes more sense to do this in the standard org manner rather than the ikiwiki manner (with a backslash in front of the link). However, the link must be protected from ikiwiki interpreting it, so things might get a little complex there.

After those two things are completed, however, I think I'm going to call this project done unless I hear feedback from users.

Some of the nicest views in Southern Maine came on the next day. Rather early in the morning, I caught this photo:

We were glad that we had waited until after the rain had stopped to climb Saddleback, which I think is the mountain that can be seen here:

First of all, it would have been cold and unpleasant if it had been raining, but more than that, we got some great views. Here is the view from the top of Saddleback:

After descending Saddleback, we got to a road crossing near Rangeley, Maine. We got a ride to the store from a person who had completed his hike a little earlier and was helping hikers on his way back south. After we had gotten back on the trail, Grinner realized that she had left her hiking poles in the trail angel's car. This was a cause for some consternation, but we had to keep pushing on. Picker simply lent Grinner one of his poles, and we kept on hiking.

Our hike that evening led us to a pond with a canoe docked on it for hikers to use. We took the canoe for a spin and filtered some water from the pond (though pond water is generally not very good).

After our little paddle, we had a nice weather for most of the next day. It was a good thing, too, because after a fairly easy morning, we had a highly treacherous descent followed by a steep ascent. By the time we had reached the shelter, it was raining again.

When I awoke the next day, the clouds had dropped and fog had set in. Luckily, Bemis mountain doesn't have any views to speak of anyway. However, it does have one feature of the mountains in Southern Maine: trail that is a fairly sheer slab of rock, often with a stream running down it. This is, needless to say, difficult to walk on. I slipped multiple times before lunch, and was not having a great day. On top of that, after lunch, there were two big, steep climbs.

At the end of the day, we got to a lean-to which contained a hiker who said that he had simply stayed there all day. He had some rather depressing stories about how hard Southern Maine had been, but in a way, it cheered me up. I could really see that others (even Northbounders, who had already come more than 1900 miles) were having trouble too. As long as I wasn't alone, I knew I could make it.

Our plan the next day was to make it to Baldpate Lean-to, just short of Grafton Notch (which contains the southernmost road that the trail crosses in Maine). However, as we were nearing the shelter, we came across a group of teenagers. We went to the shelter and found an even larger group of teens. They had completely taken over the shelter and were engaged in several loud games, completely disturbing the peace of the outdoors. In fact, the object of one of the games was to scream for the longest. It was terrible.

Luckily for me, I had been lobbying to go on to Grafton Notch for most of the day anyway, so I was pleased when Picker and Grinner got fed up with the teens and decided to go on. We made it to Grafton Notch and found some trail magic. I don't think I've described what trail magic is yet. Sometimes people who live near the trail will leave food or drink on the trail for hikers. In this case, a kind trail angel had driven his truck to Grafton Notch and was sitting with a cooler full of Nestle chocolate milk. I don't think I have ever been so happy to drink milk, and it started a tradition for me of having a chocolate milkshake whenever possible in town.

We hiked about halfway up the next mountain after our refreshment and set up camp for the night in the middle of the woods. This is known as "stealth camping".

We finished climbing the mountain the next morning, and went in the fog past Speck Pond. It made for a couple of rather pretty pictures of some lilypads:

Going down from the pond, we had more trails on slippery sheer rocks. Grinner got stuck for a couple of minutes at one point and I had to help her get down.

At the bottom of the hill, we had finally reached Mahoosuc Notch. Unfortunately I don't have any pictures of it (I was trying to remain as focussed as possible), because it is one of the most interesting features of the trail. It looks like someone took a bunch of house-sized boulders and shook them so that they landed randomly at the bottom of a valley. There are some pictures at the wikipedia page that give a good impression of what it is like. Note especially the arrow painted on the rock – those are essential to making it through in a reasonable amount of time.

In any case, we did the one mile through the notch in about two hours. For me, it was not the most enjoyable: I took a bad fall right at the entrance to the notch, so my hands were hurting from stopping the fall, and I was a bit shaken up. It was all I could do to keep my concentration and not fall again.

Finally, we got through the notch and I found that I was out of water. There was no choice but to go on to the shelter to get water. I hiked up the hill as quickly as I could, and fortuitously found a water bottle that someone else had dropped. Not knowing the provenance of the water in the bottle, I wasn't able to drink from it, but it would come in handy later in the night.

I set up my hammock and Picker and Grinner decided to stay in the shelter. It was their first time staying in a shelter with other people (they had tented until then), and Picker came over to gripe about it a bit. After supper, I went to throw my bear bag. At that point, I still wasn't great at it (and I guess I'm still not), but one throw was so unfortunate as to be comical. I had attached my water bottle to the line that would hold the bag up on the branch. On this particular throw, the water bottle arced behind me and came straight down on a sharp rock, splitting into multiple pieces. I was glad I had found the other water bottle, but it was a pretty bad end to a rather unpleasant day, which should have been a lot more fun.

The trail away from the shelter was nearly as hard as the trail to it. There were ladders and rocks to climb up. Early in the morning, as I was trying to climb up one of these rocks, I lost my grip and fell again – this time five to ten feet. I was extremely lucky not to get hurt. I think the weight of my backpack caused me to fall backwards onto it, keeping me from being too badly injured. However, I struck my jaw on a rock on the way down, and it would be sore for a couple of days afterwards. After bad falls two days in a row, I was eager to get to town.

First, though, we needed to cross the border between Maine and New Hampshire. Here is a picture proving we did it:

We didn't make it to town that night, but we made it within about six miles. When we finally got to Gorham, NH the next morning, things were already seeming better to me. We were in town and could hang out in a hostel for a fairly cheap zero day. Also, the trail angel in whose car Picker had left her poles had left them at the hostel, so she got those back. There was a place for me to get a milkshake. So at least for a day and a half of not-hiking, I was a pretty happy hiker.

Just past Gorham is the beginning of the White mountains. I will go off on my own in one of the toughest sections of the trail.