I’m quite impressed with how easy libjit is to use. I’ve been working on an instruction level simulator for a processor which will probably end up having two versions: a fast one for testing applications and an accurate one for testing the peripherals. The direct threaded version runs at a quite usable virtual 44 MHz on my 2.4 GHz Core 2 Duo. I wrote a test application in libjit and actually had trouble getting any output – the JIT recognised the unused values and kept optimising my little examples away 
I’m not sure yet how to map the assembly language into JITed functions. The obvious way is to map any basic block (a sequence of instructions that contains no branches) to a function but the basic blocks are on average only five or six instructions – the JIT overhead might cancel out any gain. The translator could recognise local jumps which would catch loops and most in-function conditionals. These only work when the jump target is known though – the target has to be an absolute immediate or relative immediate jump.
The JITed code would have many advantages. The dispatch part between instructions disappears and, perhaps more interestingly, any unused condition flag calculations will get optimised out. An example is the ADD R10, R11 instruction which is equivalent to result = R10 + R11; C = result < R10; R10 = result. If a following instruction clobbers C before it is used then the expensive less than should get optimised out.