University of Tokyo and JRuby Team to Collaborate on MVM

Finally the full announcement is available. We're pretty excited about this one...

The University of Tokyo and Sun Microsystems Commence Joint Research Projects on High Performance Computing and Web-based Programming Languages

Improving Efficiency and Simplicity of Fortress, Ruby and JRuby Languages is Current Focus for Unique Academic-Corporate Collaborative Model

TOKYO and SANTA CLARA, Calif. -- February 27, 2008 -- The University of Tokyo and Sun Microsystems, Inc. (Nasdaq: JAVA) today announced two joint research projects that will focus on High-Performance Computing (HPC) and Web-based programming languages.

...

The two research topics are:

• Development of a library based on skeletal parallel programming in Fortress
• Implementation of a multiple virtual machine (MVM) environment on Ruby and JRuby

Some of you knew about this from Tim Bray's rather cryptic announcement at RubyConf. The basic idea here is that we on the JRuby team and folks in Professor Ikuo Takeuchi's group at the University of Tokyo will be cooperating to come up with an MVM specification and implementations for the Ruby 1.9 and JRuby codebases. Tom and I have had brief discussions with Professor Koichi Sasada (ko1, creator of YARV) about this already, and there are mailing lists and wikis already running. I've also made sure that Evan Phoenix, of Rubinius, is included in the discussions. Like JRuby, Rubinius largely has MVM features implemented, but the eventual form they'll take and API they'll present is up in the air.

This should be an exciting collaboration between Sun and U Tokyo, as well as between the three most functional next-generation Ruby implementations. I'll certainly keep you posted on all MVM developments as they come up, and I welcome any comments or suggestions you might have.

JRuby in Google Summer of Code 2008

Greetings!

Google's Summer of Code for 2008 is starting up again, and we're looking for folks to submit proposals. The JRuby Community or Sun or me or someone will sign up as a mentoring organization, so start thinking about or discussing possible proposals. Here's a few ideas to get you started (and hopefully, there are many other ideas out there not on this list):

• Writing a whole suite of specs for current Java integration behavior...and then expanding that suite to include behavior we want to add. This work would go hand-in-hand with a rework of Java Integration we're likely to start soon.
• Collect and help round out all the profiling/debugging/etc tools for JRuby and get them to final releasable states. There's several projects in the works, but most of them are stalled and folks need better debugging. This project could also include simply working with existing IDEs (NetBeans, etc) to figure out how to get them to debug compiled Ruby code correctly (currently they won't step into .rb files).
• Continue work on an interface-compatible RMagick port. There's already RMagickJR which has a lot of work into it, but nobody's had time to continue it. A working RMagick gem would ease migration for lots of folks using Ruby.
• Putting together a definitive set of fine and coarse-grained benchmarks for Rails. JRuby on most benchmarks has been faster than Ruby 1.8.6...and yet higher-performance Rails has been elusive. We need better benchmarks and better visibility into core Rails. Bonus work: help nail down what's slower about JRuby.
• Survey all existing JRuby extensions and put together an official public API based on core JRuby methods they're using. This would help us reduce the hassle of migrating extensions across JRuby versions.

Please, anyone else who has ideas, feel free to post them here or on the mailing list for discussion. And if you have a proposal, go ahead and mail the JRuby dev list directly.

Update: Martin Probst added this idea in the comments:

Another idea might be to "fix" RDoc, whatever that means. That's not really JRuby centric, but still a very worthwhile task, I think.

A documentation system that allows easy doc writing (Wiki-alike) and provides a better view on the actual functionality you can find in a certain instance would be really helpful. Plus a decent search feature.

And that makes me think of another idea:

• Add RDoc comments to all the JRuby versions of Ruby core methods, and get RI/RDocs generating as part of JRuby distribution builds. Then we could ship RI and have it work correctly. Ola Bini has already started some of the work, creating an RDoc annotation we can add to our Ruby-bound methods.

Update 2: sgwong in the comments suggested implementing the win32ole library for JRuby. This would also be an excellent contribution, since there's already libraries like Jacob to take some of the pain out of it, and it would be great to have it working on JRuby. And again, this makes me think of an few additional options:

• Implement a full, compatible set of Apple Cocoa bindings for JRuby. You could use JNA, and I believe there's already Cocoa bindings for Java you could reuse as well, but I'm not familiar with them.
• Complete implementation of the DL library (Ruby's stdlib for programmatic loading and calling native dynamic libraries) and/or Rubinius's FFI (same thing, with a somewhat tidier interface). Here too there's lots of help: I've already partially implemented DL using JNA, and it wouldn't be hard to finish it and/or implement Rubinius's FFI. And implementing Rubinius's FFI would have the added benefit of allowing JRuby to share some of Rubinius's library wrappers.
  

Ruby's Thread#raise, Thread#kill, timeout.rb, and net/protocol.rb libraries are broken

I'm taking a break from some bug fixing to bring you this public service announcement:

Ruby's Thread#raise, Thread#kill, and the timeout.rb standard library based on them are inherently broken and should not be used for any purpose. And by extension, net/protocol.rb and all the net/* libraries that use timeout.rb are also currently broken (but they can be fixed).

I will explain, starting with timeout.rb. You see, timeout.rb allows you to specify that a given block of code should only run for a certain amount of time. If it runs longer, an error is raised. If it completes before the timeout, all is well.

Sounds innocuous enough, right? Well, it's not. Here's the code:

def timeout(sec, exception=Error)
  return yield if sec == nil or sec.zero?
  raise ThreadError, "timeout within critical session"\
                                      if Thread.critical
  begin
    x = Thread.current
    y = Thread.start {
      sleep sec
      x.raise exception, "execution expired" if x.alive?
    }
    yield sec
    #    return true
  ensure
    y.kill if y and y.alive?
  end
end

So you call timeout with a number of seconds, an optional exception type to raise, and the code to execute. A new thread is spun up (for every invocation, I might add) and set to sleep for the specified number of seconds, while the main/calling thread yields to your block of code.

If the code completes before the timeout thread wakes up, all is well. The timeout thread is killed and the result of the provided block of code is returned.

If the timeout thread wakes up before the block has completed, it tells the main/calling thread to raise a new instance of the specified exception type.

All this is reasonable if we assume that Thread#kill and Thread#raise are safe. Unfortunately, they're provably unsafe.

Here's a reduced example:

main = Thread.current
timer = Thread.new { sleep 5; main.raise }
begin
  lock(some_resource)
  do_some_work
ensure
  timer.kill
  unlock_some_resource
end

Here we have a simple timeout case. A new thread is spun up to wait for five seconds. A resource is acquired (in this case a lock) and some work is performed. When the work has completed, the timer is killed and the resource is unlocked.

The problem, however, is that you can't guarantee when the timer might fire.

In general, with multithreaded applications, you have to assume that cross-thread events can happen at any point in the program. So we can start by listing a number of places where the timer's raise call might actually fire in the main part of the code, between "begin" and "ensure".

1. It could fire before the lock is acquired
2. It could fire while the lock is being acquired (potentially corrupting whatever resource is being locked, but we'll ignore that for the moment)
   
3. It could fire after the lock is acquired but before the work has started
4. It could fire while the work is happening (presumably the desired effect of the timeout, but it also suffers from potential data corruption issues)
5. It could fire immediately after the work completes but before entering the ensure block

Other than the data corruption issues (which are very real concerns) none of these is particularly dangerous. We could even assume that the lock is safe and the work being done with the resource is perfectly synchronized and impossible to corrupt. Whatever. The bad news is what happens in the ensure block.

If we assume we've gotten through the main body of code without incident, we now enter the ensure. The main thread is about to kill the timeout thread, when BAM, the raise call fires. Now we're in a bit of a predicament. We're already outside the protected body, so the remaining code in the ensure is going to fail. What's worse, we're about to leave a resource locked that may never get unlocked, so even if we can gracefully handle the timeout error somewhere else, we're in trouble.

What if we move the timer kill inside the protected body, to ensure we kill the timer before proceeding to the lock release?

main = Thread.current
timer = Thread.new { sleep 5; raise }
begin
  lock(some_resource)
  do_some_work
  timer.kill
ensure
  unlock_some_resource
end

Now we have to deal with the flip side of the coin: if the work we're performing raises an exception, we won't kill the timer thread, and all hell breaks loose. Specifically, after our lock has been released and we've bubbled the exception somewhere up into the call stack, BAM, the raise call fires. Now it's anybody's guess what we've screwed up in our system. And the same situation applies to any thread you might want to call Thread#raise or Thread#kill against: you can't make any guarantees about what damage you'll do.

There's a good FAQ in the Java SE platform docs entitled Why Are Thread.stop, Thread.suspend, Thread.resume and Runtime.runFinalizersOnExit Deprecated?, which covers this phenomenon in more detail. You see, in the early days of Java, it had all these same operations: killing a thread with Thread.stop(), causing a thread to raise an arbitrary exception with Thread.stop(Throwable), and a few others for suspending and resuming threads. But they were a mistake, and in any current Java SE platform implementation, these methods no longer function.

It is provably unsafe to be able to kill a target thread or cause it to raise an exception arbitrarily.

So what about net/protocol.rb? Here's the relevant code, used to fill the read buffer from the protocol's socket:

def rbuf_fill
  timeout(@read_timeout) {
    @rbuf << @io.sysread(8196)
  }
end

This is from JRuby's copy; MRI performs a read of 1024 bytes at a time (spinning up a thread for each) and Rubinius has both a size modification and a timeout.rb modification to use a single timeout thread. But the problems with timeout remain; specifically, if you have any code that uses net/protocol (like net/http, which I'm guessing a few of you rely on) you have a chance that a timeout error might be raised in the middle of your code. You are all rescuing Timeout::Error when you use net/http to read from a URL, right? What, you aren't? Well you better go add that to every piece of code you have that calls net/http, and while you're at it add it to every other library in the world that uses net/http. And then you can move on to the other net/* protocols and third-party libraries that use timeout.rb. Here's a quick list to get you started.

Ok, so you don't want to do all that. What are your options? Here's a few suggestions to help you on your way:

1. Although you don't have to take my word for it, eventually you're going to have to accept the truth. Thread#kill, Thread#raise, timeout.rb, net/protocol.rb all suffer from these problems. net/protocol.rb could be fixed to use nonblocking IO (select), as could I suspect most of the other libraries, but there is no safe way to use Thread#kill or Thread#raise. Let me repeat that: there is no safe way to use Thread#kill or Thread#raise.
2. Start lobbying the various Ruby implementations to eliminate Thread#kill and Thread#raise (and while you're at it, eliminate Thread#critical= as well, since it's probably the single largest thing preventing Ruby from being both concurrently-threaded and high-performance).
3. Start lobbying the library and application maintainers using Thread#kill, Thread#raise, and timeout.rb to stop.
4. Stop using them yourself.

Now I want this post to be productive, so I'll give a brief overview of how to avoid using these seductively powerful and inherently unsafe features:

• If you want to be able to kill a thread, write its code such that it periodically checks whether it should terminate. That allows the thread to safely clean up resources and prepare to "die itself".
• If you need to time out an operation, you're going to have to find a different way to do it. With IO, it's pretty easy. Just look up IO#select and learn to love it. With arbitrary code and libraries, you may be able successfully lobby the authors to add timeout options, or you may be able to hook into them yourself. If you can't do either of those...we'll, you're SOL. Welcome to threads. I hope others will post suggestions in the comments.
• If you think you can ignore this, think again. Eventually you're going to get bitten in the ass, be it from a long-running transaction that gets corrupted due to a timeout error or a filesystem operation that wipes out some critical file. You're not going to escape this one, so we should start trying to fix it now.

I'm hoping this will start a discussion eventually leading to these features being deprecated and removed from use. I believe Ruby's viability in an increasingly parallel computing world depends on getting threading and concurrency right, and Thread#raise, Thread#kill, and the timeout.rb library need to go away.

FOSDEM

It's 5am in Brussels and I'm awake. That can only mean one thing. Time to blog!

This weekend I'm presenting JRuby at FOSDEM, the "Free and Open Source Developers European Meeting." I was invited to talk, and who could pass up an invitation to Belgium?

I've been trying to shake things up with my recent JRuby talks. At Lang.NET, I obviously dug into technical details a lot more because it was a crowd that could stomach such things. At acts_as_conference, I threw out the old JRuby on Rails demo and focused only on things that make JRuby on Rails different (better) than classic Ruby on Rails development, such as improved performance, easier deployment, and access to a world of Java libraries. And FOSDEM will include all new content as well: I'm spending Friday putting together a talk that discusses JRuby capabilities and status while simultaneously illustrating the impact community developers have had on the project. After all, it's an OSS conference, so I'll continue my recent trend and try to present something directly on-topic.

For those of you unable to attend, it turns out there will be a live video stream of some of the talks, including the whole Programming Languages track. I don't think I've ever been livestreamed before.

JRuby RC2 Released; What's Next?

Today, Tom got the JRuby 1.1 RC2 release out. It's an astounding collection of performance improvements and compatibility fixes. Here's Tom's JRuby 1.1 RC2 announcement.

Let's recap a little bit:

• There have been additional performance improvements in RC2 over RC1. Long story short, performance of most trivial numeric benchmarks approaches or exceeds Ruby 1.9, while most others are on par. So in general, we have started using Ruby 1.9 performance as our baseline.
• Unusual for an RC cycle are the 260 bugs we fixed since RC1. Yes, we're bending the rules of RCs a bit, but if we can fix that many bugs in just over a month, who can really fault us for doing so? Nobody can question that JRuby is more compatible with Ruby 1.8.6 than any other Ruby implementation available.
• We've also included a number of memory use improvements that should reduce the overall memory usage of a JRuby instance and perhaps more importantly reduce the memory cost of JIT compiled code (so called "Perm Gen" space used by Ruby code that's been compiled to JVM bytecode). In quick measurements, an application running N instance of JRuby used roughly 1/Nth as much perm gen space.

So there's really two big messages that come along with this release. The first is that JRuby is rapidly becoming the Ruby implementation of choice if you want to make a large application as fast and as scalable as possible. The thousands of hours of time we've poured into compatibility, stability, and performance are really paying off. The second message comes from me to you:

If you haven't looked at JRuby, now's the time to do so.

All the raw materials are falling into place. JRuby is ready to be used and abused, and we're ready to take it to the next level for whatever kinds of applications you want to throw at it. And not only are we ready, it's our top priority for JRuby. We want to make JRuby work for you.

And that brings me to the "What's Next" portion of this post. Where do we go from here?

We've got a window for additional improvements and fixes before 1.1 final. That much is certain, and we want to fill that time with everything necessary to make JRuby 1.1 a success. So we need your help...find bugs, file reports, let us know about performance bottlenecks. And if you're able, help us field those reports by fixing issues and contributing patches.

But we also have a unique opportunity here. JRuby offers features not found in any other Ruby implementation, features we've only begun to utilize:

Inside a single JVM process, JRuby can be used to host any number of applications, scaling them to any number of concurrent users.

This one applies equally well to Rails and to other web frameworks rapidly gaining popularity like Merb. And new projects like the GlassFish gem are leading the way to simple, scalable, no-hassle hosting for Ruby web applications. But we're pretty resource-limited on the JRuby project. We've got two full-time committers and a handful of part-timers and after-hours contributors pouring their free time into helping out. For JRuby web app hosting to improve and meet your requirements, we're going to need your use cases, your experience, and your input. We're attempting to build the most scalable, best-performing Ruby web platform with JRuby, but we're doing it in true OSS style. No secrets, no hidden agendas. This is going to be your web platform, or our efforts are wasted. So what should it look like?

The GlassFish gem is the first step. It leverages some of the best features of the Java platform: high-speed asynchronous IO, native threading, built-in management and monitoring, and application namespace isolation (yes, even classloaders have a good side) to make Ruby web applications a push-button affair to deploy and scale. With one command, your app is production-ready. No mongrel packs to manage, no cluster of apps to monitor, and no WAR file relics to slow you down. "glassfish_rails myapp" and you're done; it's true one-step deployment. Unfortunately right now it only supports Rails. We want to make it not only a rock-solid and high-performance Rails server, but also a general-purpose "mod_ruby" for all Ruby web-development purposes. It's the right platform at the right time, and we're ready to take it to the next level. But we need you to try it out and let us know what it needs.

JRuby's performance regularly exceeds Ruby 1.8.6, and in many cases has started to exceed Ruby 1.9.

At this point I'm convinced JRuby will be able to claim the title of "fastest Ruby implementation", for some definition of "Ruby". And if we're not there yet, we will be soon. With most benchmarks meeting or exceeding Ruby 1.8.6 and many approaching or exceeding Ruby 1.9 we're feeling pretty good. What I've learned is that performance is important, but it's not the driving concern for most Ruby developers, especially those building web applications usually bounded by unrelated bottlenecks in IO or database access. But that's only what I've been able to gather from conversations with a few of you. Is there more we need to do? Should we keep going on performance, or are there other areas we should focus on? Do you have cases where JRuby's performance doesn't meet your needs?

JRuby performance future is largely an open question right now. A stock JRuby install performs really well, "better enough" that many folks are using it for performance-sensitive apps already. We know there's bottlenecks, but we've solving them as they come up, and we're on the downward slope. Outside of bottlenecks, do we have room to grow? You bet we do. I've got prototype and "experimental" features already implemented and yet to be explored that will improve JRuby's performance even more. Of course there's always tradeoffs. You might have to accept a larger memory footprint, or you may have to turn off some edge-case features of Ruby that incur automatic performance handicaps. And some of the wildest improvements will depend on dynamic invocation support (hopefully in JDK 7) and a host of other features (almost certain to be available in the OpenJDK "Multi-language VM" subproject). But where performance improvements are needed, they're going to happen, and if I have any say they're going to happen in such a way that all other JVM languages can benefit as well. I'm looking to you folks to help us prioritize performance and point us in the right direction for your needs.

JRuby makes the JVM and the Java platform really shine, with excellent language performance and a "friendlier" face on top of all those libraries and all that JVM magic.

I think this is an area we've only just started to realize. Because JRuby is hosted on the JVM, we have access to the best JIT technology, the best GC technology, and the best collection of libraries ever combined into a single platform. Say what you will about Java. You may love the language or you may hate it...but it's becoming more and more obvious that the JVM is about a lot more than Java. JRuby is, to my knowledge, the only time a non-JVM language has been ported to the JVM and used for real-world, production deployments of a framework never designed with the JVM in mind. We've taken an unspecified single-implementation language (Matz's Ruby) and its key application framework, (Rails) and delivered a hosting and deployment option that at least parallels the original and in many ways exceeds it. And this is only the first such case...the Jython guys now have Django working, and it's only a matter of time before Jython becomes a real-world usable platform for Python web development. And there's already work being done to make Merb--a framework inspired by Rails but without many of Rails' warts--run perfectly in JRuby. And it's all open source, from the JVM up. This is your future to create.

I think the next phase for JRuby will bring tighter, cleaner integration with the rest of the Java platform. Already you can call any Java-based library as though it were just another piece of Ruby code. But it's not as seamless as we'd like. Some issues, like choosing the right target method or coercing types appropriately, are issues all dynamic languages face calling into static-typed code. Groovy has various features we're likely to copy, features like explicit casting and coercing of types and limited static type declaration at the edges. Frameworks like the DLR from Microsoft have a similar approach, since they've been designed to make new languages "first class" from day one. We will work to find ways to solve these sorts of problems for all JVM languages at the same time we solve them for JRuby. But there's also a lot that needs to come from Ruby users. What can we do to make the JVM and all those libraries work for you?

I guess there's a simple bottom line to all this. JRuby is an OSS project driven mostly by community contributors (5 out of 8 committers working in their free time and hundreds of others contributing patches and bug reports), based on an OSS platform (not only OpenJDK, but a culture of Free software and open source that permeates the entire Java ecosystem), hosting OSS frameworks and libraries (Rails, Merb, and the host of other apps in the Ruby world). All this is meaningless without you and your applications. We're ready to pour our effort into making this platform work for you. Are you ready to help us?

Lang.NET 2008: Day 1 Thoughts

Yes friends, I'm at Microsoft's Lang.NET symposium this week. Does this strike you as a bit peculiar?

Lang.NET is Microsoft's event for folks interested in using and implementing "alternative" languages on the CLR. From the Lang.NET site itself:

Lang .NET 2008 Symposium is a forum for discussion on programming languages, managed execution environments, compilers, multi-language libraries, and integrated development environments. This conference provides an excellent opportunity for Programming Language Implementers and Researchers from both industry and academia to meet and share their knowledge, experience, and suggestions for future research and development in the area of programming languages

Of course .NET and the CLR aren't mentioned explicitly here, but being a Microsoft event I think it's reasonable to expect it to be .NET heavy. And that's certainly ok, since at least one of the key professed design goals of the CLR is to support multiple languages, and by multiple we'd like to think they mean more than VB or C#. Microsoft would understandably like to cultivate a culture of language implementation and experimentation on CLR, since history has shown repeatedly that programming language diversity is a must for any platform, system, or framework to be long-lived. This fact is not new to many of us, although there are certain companies that until recently refused to believe it. But I digress.

What is a long time Java user and short time JVM language implementer doing here? Why would I subject myself to the slings and arrows of a potentially unfriendly .NET crowd, or at least subject myself to the scalding apathy of a roomful of engineers with no interest in the JVM or my work on JRuby? Well, that's an excellent question.

Perhaps it's more entertaining to describe *what* I'm doing here before explaining *why* I'm doing it. The credit for the idea goes to Brian Goetz, a coworker of mine at Sun and currently an engineer working on JavaFX (though you probably know him better from his talks on Java performance or his book and talks on Java concurrency). Given that we're both working on JVM language-related projects, and we both would like to see the JVM and the Java platform continue to grow, evolve, and expand, he thought it might be useful for us two and John Rose to attend the conference to present and discuss the many enhancements we're considering to support non-Java languages. You may know John Rose from his posts on adding various wild features to the JVM...features that you, dear reader, and others like you may never have expected to enter the JVM in any form. That's how I know him, and from my many discussions with him about how to make JRuby perform well on current JVM versions. But we must look to the future, and attending Lang.NET is at least in part to help us validate that this is an appropriate future to pursue.

So today John Rose and I presented his Da Vinci Machine project to a largely .NET crowd, with John covering the theoretical aspects of what features we'd like to add to the JVM and how he plans to add them, and me covering the practical reasons why we'd like to add those features using JRuby as an example. Those of you who saw my presentation at RubyConf would have recognized my slides; I largely presented the same "JRuby design internals" content in more depth and with language implementers and enthusiasts in mind. Specifically I presented details about JRuby's parser, interpreter, core class implementations, extensions and POSIX support, and of course its compiler and related optimizations. The last area was obviously the most interesting for this crowd, but I'll explain that later.

So there's the primary "what" of the trip. And it was the primary "what" we three shared in common. But there were a few other "what"s I had in mind entering into this, and they also explain the "why":

1. I hoped to understand better how the DLR (and to a lesser extent the CLR) improve the language-implementing ecosystem on .NET.
2. I wanted to meet and talk with .NET language implementers about their strategies for various problems facing JRuby and similar JVM languages.
3. I wanted to meet my counterparts on the .NET side of the world, which is increasingly becoming a parallel universe to the JVM and the Java platform...at least as far as problem-solving strategies and future directions go.
4. I enjoy meeting and talking with smart people. There's a few of them here.

So in order to "yeggefy" my post a bit, I want to elaborate on the first and primary point in more detail, rather than give another yawner of a conference play-by-play.

Pain

It's worth prefacing this with a disclaimer: I am not a Microsoft fan. Perhaps it's like being burned too many times by a hot iron. My work on LiteStep, my enthusiasm for open source, my interest in collaborative, communal development processes, and my dark days learning Win32, MFC, and COM mean I'll probably never be particularly warm to Microsoft projects, technology, or ideals, regardless of how they may evolve over time. That's not to say Microsoft can't change or isn't changing...but as you probably know it's hard to make yourself touch a cold iron. Pain rewires our circuitry, and lingers on in our altered behavior forever. I have felt pain.

But I endeavor to rise above my own wiring and bigotry, which is why I force myself to help other JVM language implementations, force myself to help other Ruby implementations, and force myself to share freely and openly all I can even with members of otherwise competing worlds. In this sense, the pain that comes from cooperating with projects I might otherwise want to undermine, derail, or discount is antithetical to my believe in open doors and open processes. I must cure myself of this affliction, rewrite myself to not feel that pain.

And so I am trying to study the DLR, which brings me to the big point #1. I will appreciate any and all corrections to everything I state here.

Eyeing the DLR

The DLR is Microsoft's Dynamic Language Runtime, a set of libraries and tools designed to make it easier to implement dynamic languages atop the CLR. The DLR provides facilities for compiler and interpreter generation (via language-agnostic expression trees), fast dynamic invocation (via self-updating dynamic call sites), and cross-language method dispatch and type system support. It is, to my eyes, intended to be the "everytool" needed to implement dynamic languages for .NET.

The DLR has largely grown out of Jim Hugunin's work on IronPython, and aims to tackle all the same issues we've dealt with while working on JRuby. But it also provides more than that in its self-optimizing capabilities and its expression tree logic, two features that undoubtedly make it easier to produce CLR dynamic language implementations with acceptable to excellent performance characteristics.

Expression trees in the DLR are used as an intermediate representation of language semantics. Where the typical language implementation process has you parse to an AST and then either interpret that or perform the additional work to compile the AST to a target "assembly" language, expression trees are meant to be (mostly) the last step in your language implementation process (outside of implementing language-specific types and dealing with language-specific nuances). Up to the point of creating a parsed AST the two paths are largely the same. But on the DLR, instead of proceeding to compiler work, you further transform the AST into an expression tree that represents the language behavior in DLR terms. So if your language provides mechanisms to conditionally run a given expression (modifiers in Ruby terms), you might turn your one conditional expression AST node into equivalent test and execute DLR expression tree nodes. It appears the translation is almost always a widening translation, where the DLR hopes to provide a broad set of "microsemantics" that can be used to compose a broader range of language features. And of course the idea is that there's a quickly-flattening asymptotic curve to the number of expression tree nodes required to implement each new language. Whether that's the case is yet to be seen.

I must admit, I've poo-pooed the expression tree idea in the past, mostly because I did not understand it and had not found a way to see beyond my personal biases. While a language-generic expression tree is certainly a very appealing goal, one that lowers the entry cost of language implementation considerably, it always seemed like a too-lofty ideal. Of course it's impossible to predict what features future languages might decide to add, so can we reasonably expect to provide a set of expression nodes that can represent those languages? Won't we be endlessly forced to extend that set of node types for each new language that comes along? And if to alleviate such pain we are given hooks to call back to custom language-specific code for cases where the DLR falls down, doesn't that eliminate the advantage we hoped to gain in the first place?

I'll make no secret about my skepticism here, but I've seen a couple things today that make me far more optimistic about the utility of language-agnostic expression trees.

The first was during Anders Hejlsberg's talk about C# 3.0. C# 3.0 adds a considerable slate of features, some which I can appreciate (type inference, for example, would take 99% of the pain out of using generics; Anders called that out as a key reason for adding inference to C#) and others which I'm still dubious on (LINQ's additions to C#'s already large set of keywords). But his talk made one thing abundantly clear: these changes are little more than compiler magic, smoke and mirrors to produce syntactic sugar. And he admitted as much, showing that behind the scenes LINQ and inferred types and their ilk basically compile down to the same IL that would be produced if you write the same sugar by hand. And that's particularly heartening to me, since it means the vast majority of these features could be added to Java or supported by JVM languages without modifying the JVM itself. Whether they're features we want to add is a separate discussion.

The second detail that made me better appreciate expression trees stems directly from the first: they're just objects, man. Although expression tree support is being wired directly into C# and VB.NET for at least the small subset of expressions LINQ encompasses, in general we're talking about nothing more than a graph of objects and a set of libraries and conventions that can utilize them. If it were reasonable for us to expose JRuby's AST as a "one true AST" to represent all languages, we'd be able to support a similar set of features, adding syntax to Java to generate Ruby ASTs at runtime and capabilities for manipulating, executing, and compiling those ASTs. Naturally, we'd never try to make an argument that the JRuby AST can even come close to approximating a set of low-level features "all languages" might need to support, but hopefully you see the parallel. So again, we're working at a level well above the VM itself. That means the power of expression trees could definitely be put into the hands of JVM language developers without any modifications to existing JVMs. And in the DLR's case, they're even considering the implications of allowing new language implementations to provide extensions to the set of expression tree nodes, so at least some of them recognize that it's all just objects too...and perhaps starting to recognize that nobody will ever build the perfect mouse trap.

So that brings me to my other key point of interest in the DLR: its mechanisms for optimizing performance at runtime.

The DLR DynamicSite is basically an invokable object that represents a given call at a given position in a program's source code. When you perform a dynamic invocation, the DynamicSite runs through a list of rules, checking each in turn until it finds a match. If it finds a match, it invokes the associated target code, which presumably will do things like coerce types, look up methods, and perform the actual work of the invocation. As more calls pass through the same call site, DynamicSite will (if necessary) repeatedly replace itself with new code (new IL in fact) that includes some subset of previous rules and targets (potentially all of them if they're all still valid or none of them if they're all now invalid) along with at least one new rule for the new set of circumstances brought about by this call. I've prototyped this exact same behavior in JRuby's call sites (on a more limited scale), but have not had the time to include them in JRuby proper just yet (and might never have to...more on that later). The idea is that as the program runs, new information becomes available that will either make old assumptions incorrect or make new optimizations possible. So the DynamicSite continually evolves, presumably toward some state of harmony or toward some upper limit (i.e. "I give up").

And there's a simple reason why the DLR must do this to get acceptable performance out of languages like Python and Ruby:

Because the CLR doesn't.

A Solid Base

In the JVM world, we've long been told about and only recently realized the benefits of dynamic optimization. The JVM has over time had varying capabilities to dynamically optimize and reoptimize code...and perhaps most importantly, to dynamically *deoptimize* code when necessary. Deoptimization is very exciting when dealing with performance concerns, since it means you can make much more aggressive optimizations--potentially unsafe optimizations in the uncertain future of an entire application--knowing you'll be able to fall back on a tried and true safe path later on. So you can do things like inlining entire call paths once you've seen the same path a few times. You can do things like omitting synchronization guards until it becomes obvious they're needed. And you can change the set of optimizations applied after the fact...in essence, you can safely be "wrong" and learn from your mistakes at runtime. This is the key reason why Java now surpasses C and C++ in specific handcrafted benchmarks and why it should eventually be able to exceed C and C++ in almost all benchmarks. And it's a key reason why we're able to get acceptable performance out of JRuby having done far less work than Microsoft has done on IronPython and the DLR.

The CLR, on the other hand, does not (yet) have the same level of dynamic optimization that the JVM does. Specifically, it does not currently support deoptimizing code that has already been JITed, and it does not always (occasionally? rarely?) include type information into its consideration of how to optimize code. (Those of you more familiar with CLR internals please update or correct me on how much optimization exists today and how deep those optimizations go.) So for dynamic languages to perform well, you simply have to do additional work. You don't have a static call path you can JIT early and trust to never change. You can't bind a call site to a specific method body, knowing verifiably that it will forever be the same code called from that site. And ultimately, that means you must implement those capabilities yourself, with profiling, self-updating call sites that build up rule and target sets based on gathered information.

So I think there's a reasonably simple answer now to folks asking me if I believe we need a "DLR" for JVM language implementers to target. The answer is that certain parts of the DLR are definitely attractive, and they may be worth implementing to ease the pain of JVM language implementation or at least reduce the barrier to entry. But there's a large set of DLR features we simply don't need to create if we can find a way to open up the JVM's existing ability to optimize dynamically...if we can punch a hole in the bulky, dynamic-unfriendly Java exterior. And that's exactly what we're doing (really, John is doing) with JSR-292 (dynamic invocation for JVM) and the MLVM.

Where then do we go from here? If the correct path is only partly down DLR street, what else is missing? Well, some of my Sun brethren may not like to hear these (and some other may love to hear them), but here's a short list of what I believe needs to happen to keep the JVM relevant into the future (at least if we take it as written that multi-language support is a must):

1. We need clear and consistent proof that a multi-language JVM is a priority, and not just at Sun Microsystems but also in the wider JVM and Java platform communities. The libraries and runtimes and support code necessary will grow out of that commitment. It's obviously not worth the effort to make all this happen if nobody cares and nobody will want to use it. But I don't think that's the case, and currently there's not a strong enough indication from any of the major community players that this cause is worth fighting for. At least in Sun's case, we've thrown down the gauntlet by open-sourcing Java and initiating projects like the JVM Languages mailing list, the JVM Language Runtime project, and the Multi-Language VM project. But we need you, and that brings me to my second point.
2. We're fighting an uphill battle for talented, excited resources in this domain, and without more money and time spent by the major players on multi-language support for the JVM (in collaboration, I might add), it's not going to happen. Again, whether that's a bad thing depends on whether you believe in a multi-language world. I do, and I'm willing to spend whatever of my time is necessary to make this happen. But I'm no patsy for political interests that might route us down the wrong path, and I'm definitely not going to be able to do this alone. Where do you stand?
3. We need the freedom to make the JVM "reborn". Java is suffering from middle age (or old age, depending on who you ask) so making nontrivial changes to the JVM specification is generally met with stiff resistance. But I believe this resistance comes either from folks who don't realize the stakes involved, or from folks with their own bigotry and biases, or perhaps simply from rank-and-file pragmatists who don't want to or can't invest the resources necessary to make their implementation of the "JVM" more useful to the relative few of us currently interested in next-generation JVM language support. And of course, there were lots of folks who believed without a doubt the Titanic could never sink, and that additional resources to ensure it would be wasted. It's time to learn from such mistakes. Yes, we can and will use MLVM as a proving ground for these features, and yes, I'm sure Sun and other players will continue to use careful, measured steps to evolve the JVM and the Java language. These are both appropriate directions. But we must never shut the door on the future by claiming that either the JVM or the Java language are "done". There's three words for an entity that can no longer change: "it's dead, Jim". Java is not dead...and I'll be damned if I'm going to let it or the JVM die.

As always my opinions are my own, and may or may not reflect the positions of Sun Microsystems or its partners. But I know for a fact there are many other like-minded individuals that feel the same way.

I'm looking forward to day two of Lang.NET 2008, and to hearing from all of you out there. Comments, questions, flames and praise are welcome. But actions speak louder than words, and the time for action is now.

Update: Here's a link to Tom's and my RubyConf slides. I'll add a link to the Lang.NET slides when they're posted.

Jython's Back, Baby!

Well it's been a long hard slog for the Jython team. Once thought dead, they seemed to pick up steam more and more over the past year. They got out a long awaited 2.2 release and started to work on many missing 2.3, 2.4, and 2.5 features. They started tackling new parsers and compilers. They spoke with me and other folks at Sun about the future of dynamic languages on the JVM. And above all, they've been working their asses off, having sprints and codefests and hacking away on every corner of Jython.

It looks like their hard work is paying off. Jim Baker reports that they have successfully run Django on Jython. They're using bleeding edge revisions of both Jython and Django, and there's a bit more work to be done, but hey, lots of folks thought it would be impossible. Haven't we heard that somewhere before?

Hats off to whole Jython team and their obviously excellent community for making this a reality. This is just the beginning!