I don't necessarily disagree with what you are saying. My day job doesn't either, which is why they do indeed develop almost everything in Java.
The thing is, for my business, Ruby (via Rails and friends) was ready for production without me having to play system architect, a role that I am manifestly unsuited for. It may not be ready for different projects but, well, I have little worry about my capability to implement projects which I do not actually have to implement.
There was no fiddly messing -- two years ago when I launched my site it was just "code code, launch", the framework has gotten better (without my intervention!) over the past two years, and when I upgraded it recently from "a mostly static site with a half-baked CMS up its sleeves" to a full-blown, users actually banging on it as we speak webapp, it required fairly little pain and delivered something which exceeded my expectations. Time spent on programming in Rails reliably delivers features my users can use.
I can write Common Lisp, in a very very theoretical sense (much like my ability to write Ruby two years ago). Its just, well, I think if I started to play "find that library" I'd discover a lot of pain points. Ruby had ActiveRecord, which is a nice safe choice for an ORM that I can rely on meeting my very modest needs going forward. Common Lisp has a wealth of choices but no clear winners, and I am disturbingly certain that if I bring that up I'll be told "Yeah, but writing your own not-quite-a-database is pretty fast".
(I mean no slight against the author of a certain site which is locally popular but since I am neither a particularly good programmer nor independently wealthy any project which includes "write your own in-memory database!" fills me with existential dread. I would rather spend my limited budget of time delivering features that users will pay for. Writing databases is worth great amounts of geek cred but merely being able to persist data does not convince my users to pay me money -- it is a prerequisite, yes, but it isn't the value add or anything close to it.)
I also understand where you're coming from...we actually use Django (or other Python frameworks) for most of the simpler sites we have to do. CL can be both painful and overkill when tackling certain types of projects. We haven't built a blog engine or standard CMS in CL, for example, because there are plenty of other polished contenders. But we don't do many simple websites/projects...and CL really comes into its own when you have a more complex project to tackle. Quite a bit of what we do would be characterized as "enterprise" software with a web interface. The 'weird' CL approach (DSLs, bottom-up, context-oriented, etc.), when fully embraced, CAN provide the fabled competitive advantage. But we try to use CL only where it is wise to do so, and we don't reinvent wheels needlessly. We haven't had to write our own database, for example...we usually use Postgres. We've used other databases and 'not-quite-a-databases', but we've never had to write our own. Most of the guts of our software is written in CL, but we have called out to Java, Python, C/C++, and Erlang libraries, as well. CL is very flexible, and when there is not a decent existing library for something it is pretty easy to Borg another codebase. But you'd be surprised how much decent code in CL is already out there.
Oddly enough, I got into CL because it was basically the only language that clicked for me. I'm the last guy on earth that would be a Lisp snob, or who is going for geek cred or technical wizardry. I think you should use what works for you. I largely use Lisp because I'm self-taught and I suck as a programmer. No joke. When I was "language shopping," Lisp seemed like a loser because of the lack of libraries. But I'm a horrible programmer, and I struggled with learning most languages beyond a certain level. I struggled with syntax. I struggled with the seemingly arbitrary rules. I struggled with understanding the approach other programmers took when solving problems. I even struggled with "easy to learn" Python beyond a certain level. But once I finally tried CL, it simply made it easier for me to build abstractions I understood and to get things done. I can totally understand how weird the "NIH" culture seems from the outside, but it often IS easier to write custom-fit code for your project in CL than it is to learn someone else's general solution well enough to make it work. Sometimes things are counterintuitive...or maybe some people are just weird, I dunno? Fortunately, there are plenty of language/framework options, so most people can find something that fits.
The thing is, for my business, Ruby (via Rails and friends) was ready for production without me having to play system architect, a role that I am manifestly unsuited for. It may not be ready for different projects but, well, I have little worry about my capability to implement projects which I do not actually have to implement.
There was no fiddly messing -- two years ago when I launched my site it was just "code code, launch", the framework has gotten better (without my intervention!) over the past two years, and when I upgraded it recently from "a mostly static site with a half-baked CMS up its sleeves" to a full-blown, users actually banging on it as we speak webapp, it required fairly little pain and delivered something which exceeded my expectations. Time spent on programming in Rails reliably delivers features my users can use.
I can write Common Lisp, in a very very theoretical sense (much like my ability to write Ruby two years ago). Its just, well, I think if I started to play "find that library" I'd discover a lot of pain points. Ruby had ActiveRecord, which is a nice safe choice for an ORM that I can rely on meeting my very modest needs going forward. Common Lisp has a wealth of choices but no clear winners, and I am disturbingly certain that if I bring that up I'll be told "Yeah, but writing your own not-quite-a-database is pretty fast".
(I mean no slight against the author of a certain site which is locally popular but since I am neither a particularly good programmer nor independently wealthy any project which includes "write your own in-memory database!" fills me with existential dread. I would rather spend my limited budget of time delivering features that users will pay for. Writing databases is worth great amounts of geek cred but merely being able to persist data does not convince my users to pay me money -- it is a prerequisite, yes, but it isn't the value add or anything close to it.)