Tales of Woe
So... in an attempt to use preexisting wheels, rather than reinvent my own at every turn, I've been trying to get a decent Common Lisp environment working with the CDK (Chemistry Development Kit). My abcl-cdk adventures actually went reasonably well and I was able, eventually, to get ABCL talking nicely to CDK. Of course I wanted more than just that, I wanted interoperability between the CDK and my half-round wheel, chemicl, a cheminformatics package I started writing in Common Lisp. This is where the train began to fall of the tracks.
ABCL and cxml-stp
A while back, in an earlier, aborted attempt to get some of my chem/bioinformatics(https://github.com/slyrus/cl-bio) stuff working with ABCL I noticed that plexippus-xpath couldn't be loaded into ABCL. This was fixed, so I was encouraged that things might work with ABCL. (While I'm on a rant, the ABCL trac issue tracker is really slow...). However, cxml-stp seems to break ABCL.
Hopefully this is a fixable bug and some future version of ABCL will work with cxml-stp.
In the meantime...
SBCL and Java
So, I figured I'd try some other approaches to getting Java and a Common Lisp implementation to play nice. I know, you're thinking "why doesn't the dude just use clojure? After all, that's what clojure was designed for!" Well, that's a good question. I did use clojure for some earlier explorations with CDK and, while the java integration generally works well, I have a bunch of existing Common Lisp code I'd like to use and, at the time at least, it seemed like all of the clojure wrappers where thin wrappers around ugly Java libraries. I've grown to know and love many Common Lisp libraries, many of which are nicely available in QuickLisp, and I'd like to be able to use those (things like cxml-stp, plexippus-xpath, opticl, etc...).
Anyway, I tried to get some sort of SBCL Java interoperability working. Three possibilities appeared: 1) jfli, 2) foil and 3) cl+j. Turns out jfli is (was?) Rich Hickey's pre-clojure Common Lisp. I'm guessing that the challenges in getting jfli to work with any of reasonably Common Lisp implementations was part of the motivation behind clojure. In any event, it doesn't seem that jfli works under SBCL.
Next, I looked at foil, which appears to use sockets to communicate to another process running a JVM. This sounded suboptimal but, presumably, workable. Turns out foil looks like some sort of windows-only beast with a bunch of C# files. Not for me.
Finally, I looked at cl+j and it turns out there are some scary warning messages about how cl+j can't possibly work with SBCL's foreign threads handling mechanism. Bummer. This seems somewhat unreasonable on SBCL's part. Surely some amount of engineering should make it possible to have both a JVM and SBCLs runtime running in the same process. Unfortunately, I'm too out of practice with SBCL internals to give this much of a go at this point. Bummer again.
CCL and Java Ok, next approach. How about cl+j and Clozure Common Lisp (CCL)? Seemed reasonable, but, unfortunately, hung just like SBCL did. Presumably this is more of a MacOS issue than a CCL issue, as cl+j is supposed to work with CCL, but maybe just on other non-mac platforms.
So, it seems I'm stuck without a viable approach to using the common lisp libraries I want and the java libraries I want in the same process. Perhaps the ABCL bug will get fixed. Perhaps JVM integration would make a good summer project for the next SBCL Summer of Code.