There are $1 ARM chips available (not sure at what volume), so it isn't just high-end 8051 chips that have to fight with the ARM chips. From an economical point of view, the ARM chips are very viable to use in both cheap and expensive equipment. It is more a question of preferences and already aquired tools and skills. But more than one processor vendor or distributor will sometimes give away free demo boards, so a hobbyist should keep the eyes open for funny freebies to play around with.

I have looked a bit at the Olimex boards. If I didn't need Ethernet, they have a couple of interesting header boards - hopefully soon with some of the newer NXP chips too. That ST USB "thumb" unit looked quite nice since it had both USB and Ethernet. The only thing that remains to be seen is how many digital signals that are accessible on it.

I have access to the Keil ARM tools, but when fooling around with hobby projects it can be an idea to look at gcc or similar. That would make it easier to possibly publish any project. Not too many hobbyists can afford the Keil license, and even tiny projects will pass the limits of the evaluation version - especially if USB or Ethernet is involved.

Edit: I have also kept an eye on this board:
http://www.olimex.com/dev/lpc-2478stk.html
Only a GPS and a 3G module missing, but that shouldn't be too hard to solve since that is what I do for a living.