The Third Bit

I recently decided to use Rails for my latest home-grown project for exactly these reasons – well, perhaps less-so for the pub nights.

Adrian and Jacob is writing a django book, so there is at least one book in the python web world.

Espen: In fairness, a TurboGears book is also being written/published.

Yes, I know there’s one Django book, and one TurboGears book, in the works, and I look forward to them both — but 1+1 is much less than 12 (and counting), and “in the works” isn’t the same as “on the shelves”. Any chance that the two books could be combined into one, even if the software isn’t?

According to this logic, Rails is the success it is because it’s the only major web-app framework for Ruby. I don’t think that is true.

No, Rails is getting all the attention because it’s a very good framework and has been attracting web developers due to its quality (the very effective marketing helped a lot, of course). People have been *switching* to Ruby just because of Rails, not coming to Ruby and looking for a web framework.

Chicken and egg — Rails is a good framework in its own right, but it was also so much better that everyone doing web programming in Ruby dropped whatever they were working with and switched, which accelerated Rails’ development tremendously. If everyone building frameworks for Python were to put their effort into [name of framework goes here], things would look very different in just a few months.

I’m curious why you’re limiting it to Django and TurboGears. Zope (and Plone) probably remains the most used Python web application framework out there even if it isn’t getting the current hype. I count 16 Zope related books on the first page of results at Amazon. And there are plenty of Zope and Plone related conferences, sprints, and other get-togethers.

rails is popular because it fulfills a common need – a quick way to get basic data-driven websites up and running. if that’s not your need, the number of books on the subject is completely irrelevant.

the python frameworks address different needs. they seem to be more component-oriented, allowing one to mix-and-match in the different functional areas. as i understand it, the ease of use of rails starts trending downward once you step outside its established conventions.

i looked at rails – it wasn’t what i was after. i would be sorely disappointed if there were only one rails-like framework on the python side and it too wasn’t right for me. i’m glad there are several to choose from.

i chose pylons, btw.

I have to echo Christopher — Rails wasn’t a success because it was the only Ruby framework. At all. Ruby people jumped on the Rails bandwagon like other groups, but they didn’t *create* the Rails bandwagon. There are other reasons (which I won’t go into here, but I have explored on my blog in the past) why I think Rails had its success. Some are simply not repeatable — we can’t just take their hype and make it our own, or try to appropriate their larger narrative and apply it to Python.

Python has a problem in a certain segment — people who buy into Python, but don’t have a lot of experience, and want to do some web programming. That’s not an insubstantial problem, but it’s only one segment. And that segment actually looks *nothing* like the segment that went to Rails — people bought into *Rails*, then learned Ruby.

But what was I saying… hmm… getting a good story together for new users is important. That story probably should not start with “analyzing what framework is best for you.” But getting a story together also doesn’t happen by fiat. The Django people have shown some willingness to create that story, though they aren’t alone. And if multiple such stories exist then I don’t think that’s such a big problem — that zero such exist is of course a problem.

Django isn’t even close to where Zope was 8 years ago.

I think you might have heard of the book “The Paradox of Choice: Why More Is Less.” If not there’s a Google talk video floating around on the Net. The book’s point is competitions and more choices are generally good for people (if they decide to actually look into the choices). But most people simply hate choices. There is a point of diminishing return. After this point people simply refuses to take actions.

Too much web frameworks discourages/distracts people who have made their decision. We all know none of the web frameworks are perfect. I know it sounds stupid, but it’s true. People hate being regretful, and with so many choices (and none of them are perfect, or the opposite coin each has its sweet spot), people who have to make the choice usually anticipate the possibility that their decision is a bad one. And, they end up avoiding making the decision all together (i.e. RoR off they go).

Having plug and play components is wonderful. I love to have the ability to pick the best (or the fittest) technology I can use for my projects. But to be honest, I (nor most the developers) don’t want to make that decision ourselves hence the Paradox of Choice. For must of the developer they reason they buy into a particular platform/framework is to rid of choice. If the platform is so diversified from within (e.g. each component is not standardized and plug-n-playable), you’re most likely going to end up with many developers each knowing one or two components very well but not the others. As whole, this defeats the purpose of having a platform/framework to begin with.

I think GvR’s intention in picking Django as the “winner” is to make people gather their energy in one place. Make ONE not-so-perfect framework perfect instead of having 10 imperfect frameworks competing for developers’ attention and creating needless distractions.

having the whole of the python web framework community gather their energy behind django doesnt seem very realistic, and i doubt it was GVRs intention in declaring the “winner”. if anything, it was to prod the rest of the community to voice more of an opinion about it, and in that regard its worked terrifically.

I actually do recommend django to python (and programming) newbies since it does seem to have the lowest learning curve of them all, tons of easy documentation and a whole community that has become markedly friendlier than they used to be. That is a terrific thing. If it allows people to be exposed to Python instead of Ruby, thats just great. But beyond that, I dont get the impression that the django developers are truly open to other components and tools being swappable and just as nicely integrated as its default components.

The standard rebuttal is “you can use any ORM/template/component you want with Django”. Well of course you can, its only Python. But Christopher Lenz puts that rebuttal to rest; when you swap in other components, django’s ease ,interoperability, and practically the entire point of using it goes out the window. its not unlike Microsoft’s old claim that “internet explorer is not bundled with the OS”…nobody believed that one either.

Django’s ORM could be built directly on top of SQLAlchemy, for example….allowing its functionality to be expanded dramatically, not scaring anyone away since the API wouldnt change, and greatly improving Django’s overall interoperability with raw SQLAlchemy models. Templating could go through an adapter such as Buffet to better allow other templating systems to cleanly integrate with Django. But go through IRC logs and such and you see a real resistance to the integration of external toolsets; layers of abstraction being added that would allow the same interoperability while at the same time allowing componentization and a much more rapid expansion of Django’s potential capabilities. I dont know why this is; some theories are that they dont understand how these things could ever work, or Django’s architecture is too rigid to allow such a task to be undertaken without an immense effort, or that they want to promote some kind of component “lock-in” to help maintain their popularity. Whatever the reason is, if Django wants to take its place as the “winner”, a great responsibility comes with that. i think its time for them to be more open to allowing true integration of the ideas and techniques of the rest of the community. maybe thats what guido was getting at.

I don’t care how much Ruby or Python books are out there.
I don’t even care which language/framework has more users, or which one is has more sex appeal.
What I really want, is “availability”.

The python community should find a way to make Python easy to install and configure by hosting providers, with no more knowledge required than that for installing php, for example.

Lets make Python ubiquitous and cheap, and we will see it succeed.

Your arguments are weak.

Number of Zope books: 222 (according to amazon)
Number of Zope sprints: 40

I hereby declare Zope winner.

“some theories are that they dont understand how these things could ever work”

gotta love the “marketing by claiming that everyone else is stupid” approach.

frederik -

youre dodging the issue raised by my post by picking the one mildly unconstructive phrase out of a four paragraph post and discarding the rest. Take that phrase out of my post.

now, how about responding to the rest of it ? lets go with the assumption: all python frameworks besides Django are now “losers”, and continuing to work on them is deterimental to Python because theres “too many frameworks”. How exactly does the entire community come behind Django ? Is Django open to true integration of other ideas and components, or are we to simply accept Django’s existing components as “the only way to do it”, warts and all, and we basically throw everything else away ? Its time to put the smokescreens away and begin addressing this.

also if anyones marketing anything here, its Python itself. We all share that goal. Thats why I recommend Django to people who I know would like it; theyd have the most chance of success and therefore getting excited about Python.

As people have mentioned, Zope’s got a decent selection of books, though quite honestly, not that many current ones, though there are a couple in the works that should improve that.
I think what people who want a single framework are missing is that Python isn’t Ruby. Ruby is very convenient. It provides ways for people who understand it to get things done in a very efficient manner. Python, to my mind, is slower moving, but more focused on getting things done in way that is straightforward. From my experience with Rails, it is clearly a Ruby framework. It does many things very efficiently, and even if it doesn’t make sense to me the first couple times I do certain things, it does allow me to get things done. I think Python web frameworks as a whole are mirroring Python, in that they are all striving to provide a straightforward solution to the problems they encounter. Straightforward solutions are much slower maturing than efficient solutions. Zope3 may still be overly complicated, but there is so little magic in what it does that it makes me far more confident in the work I do in it.

Overall, I really don’t think we have anything to worry about. I see Python and Ruby continueing to grow, at whatever pace suites each of them. There are still many areas of application development for each of them to fit. If Ruby grows faster, that’s perfectly acceptable to me.
Zope is maturing, TG is maturing, Django is maturing, and there are a number of other web frameworks (or, more importantly, pieces of web frameworks) that are looking quite promising.

Mike — I think it’s important to note that, over the past year since Django has been open sourced, we’ve made a remarkable amount of changes and improvements to the framework, *all* coming from the community. Put another way: Just because Django isn’t a collection of pre-existing Python libraries doesn’t mean the framework doesn’t get contributions. The repercussions of this are particularly impressive, if I may say so, when you consider Django was a closed-source project for several years, written by a company with dozens of thousands of lines of Django code and no real incentive to encourage sweeping, backwards-incompatible changes, the likes of which we’ve seen over the past year in Django, thanks to community contributions.

But I think you’re referring to contributions from a different “community” — other framework authors, such as yourself, not the community of users of Django. To understand our perspective on that, you should understand that our goal has always been to make the best possible user experience for people using Django. It’s all about the usability. It’s not that we actively *want* to be disassociated with other frameworks (such as your own SQLAlchemy, which I think is a fantastic piece of software), it’s just that making our own framework as good as possible is Priority Number One. The fewer decisions people have to make, the better. The fewer packages people have to download, the better. The easier the installation, the better. *Those* are the core principles that have guided us — not any sort of antipathy toward other frameworks for the sake of antipathy. (We’re nice people, really! )

For instance, I think it’d be great if somebody integrated SQLAlchemy with Django, so that people could optionally take advantage of SQLAlchemy if they have it installed. Just because we haven’t done that doesn’t mean we don’t *want* to do it; it’s a matter of priorities for the core team. That translates into, “I, Adrian, would probably never use that, so I can’t justify coding it myself, but if somebody else wants to code it, feel free.” At any rate, in the wake of this brouhaha about “official Web frameworks,” I’ve set the wheels in motion regarding starting that project. Stay tuned!

Adrian -

I entirely agree, i wasnt proposing its something “Adrian should be doing !” With SQLAlchemy i am constantly walking the line between “yes, SA would like this feature/no, SA’s current developers cant take it on”, such as the ActiveMapper and Migrate packages which I’ve basically “outsourced” as things i dont need myself, but have obvious value to others.

However, I have had to take a lot of steps with SA to insure that its fully open to extensibility, including its general layout as well as various hooks (and with Myghty as well). I know the point of Django is to bring the best experience to its users, and I *totally* understand the idea of its in-house roots; ive written several (now very obsolete) frameworks where i had to invent everything. But when such a framework comes out in the open, has been around a while, and has been as successful as Django, theres a next level that needs to be considered.

So, if someone came to you with Django’s ORM built entirely on SA, which then magically knew how to do three more things that the existing ORM could not, or a template plugin that nicely integrated Kid/Markup templates as well as that of Django’s (in fact I like Markup quite a bit…), it might require some rearrangement/decoupling of Django’s internals to make it possible (from what you said in your last sentence it seems like you are considering that already). Django could also pursue the direction of rearrangement/decoupling of internals beforehand to make it easier for such projects to happen in the first place (and maybe thats starting to happen too). Also, while i know it has introduced some WSGI support, it can take hard and honest looks at extremely modular WSGI frameworks like Pylons and see what can be adopted there…since a *lot* of people have put a lot of work into WSGI and its a really good thing (GVR-approved!).

This is all based on the idea that Guido truly wants there to be just “one” framework. If that is the case, then he’s really asking us to all gather behind it. If he wants us to stop
introducing competing approaches and just pick this one, then it necessitates that this idea become prominent within the developmental direction of Django – that it graduate from its roots as a very successful “in-house get the job done” application into a much more broadly-scoped and organically-growable system. While that may seem overwhelming, if the whole of the Python community were behind it, the resources are definitely available…but only if the Django project puts out a strong and consistent message that this is what it really wants. Compared to an in-house and homegrown background, this is a major change of mindset for such a framework.

Greg, I’ve reminded you several times of a particular web framework that you apparently keep forgetting about:

Zope.

Zope has had gatherings since 2000 or thereabouts. I’ve listed the gatherings I actually attended as an eye-witness:

* IPC in 2000, Zope track

* IPC in 2001, Zope track

* european zope meeting at linuxtag in Germany, 2000

* zope track at linux conference in Amsterdam, 2001.

* 2 zope bbqs (actually 2 day conferences) in Berlin, in 2001 and 2002

* EuroPython 2002, 2003, 2004, 2005. 2006 had a
more general web track to recognize the newcomers, but all the ones before had big Zope tracks, and attracted at least one third if not more of attendees..

* Plone conference 2005 and 2006 in Vienna.

* numerous Zope sprints.

That’s about 10 gatherings, not including the sprints. Including gatherings I wasn’t at, I think that makes for 15 to 20 major Zope gatherings (at least one full track lasting at least 1 days, though mostly more) in the last 6 years or so. There have been at least that many sprints – many more, I suspect.

As you know, there are quite a few books out about Zope as well, though admittedly many of them are not that current. That happens when you’ve been doing cutting-edge web development in Python for the last 7 years that Zope has been open sourced.

Anyway, it’s clear Zope needs to drastically improve its marketing, a major lesson we’ve been too slow to learn. We’re learning it.

I’ve talked to you about Zope several times in this context, and yet you keep forgetting about it – I’m getting a bit worried, are you all right?

Martijn inadvertently makes a point: previous EuroPython conferences had their own Zope track, and it seemed almost like a separate conference, although Martijn himself obviously participated in more than just the Zope track. And where Martijn says that such tracks attracted numerous attendees, my impression was that those attendees were mostly there for Zope stuff, not for Python stuff – in 2005, the Zope track even had its own lightning talk session running parallel to the Python one, making it difficult to put a finger on the pulse of both communities unless one was willing to spend all one’s time running between lecture theatres. Such planning makes one wonder how interested the average Zope person is about general happenings in the Python scene and vice versa.

It’s clear that Zope is considered to have a community of its own, and the EuroPython track organisation of previous years merely confirms that people on both sides of the fence realise this. Zope experts seem to have no problems in “jumping the fence” and applying Python solutions within the Zope architecture, but Zope non-practitioners look at the height of the fence and question Zope’s relevance. One observation I see occasionally is that such people “just don’t get it”, despite there having been plenty of valid criticisms of Zope over the years, and to not take those criticisms seriously just adds to that credibility problem, good marketing or otherwise.

To clear one thing up: I believe that the Zope track particularly attracted a lot of attendees to previous EuroPython conferences, and that quite some of these people were there for Zope-related matters primarily. So no need to put a different perspective on it: that’s my perspective already. One of my hopes always was that this would help integrate the Zope community with the Python community better, as people would inevitably interact, and the Zope people would hopefully go see some Python-related talks as well.

Zope clearly does have a community of its own. What is the problem with that? Surely say, Django, has a community of its own as well?

Zope may be slightly different in that it attracted many people to Zope for Zope primarily as opposed to through Python, but that’s one of the things people say about Ruby on Rails too. Another difference for Zope is that it traditionally attracted many casual programmers and non-programmers to the system as well. I think that overall this enriched the Zope community, but it is indeed a unique challenge to deal with.

Zope’s community of course also clearly has large overlaps with the Python community. The Zope community believes much good can be learned from the wider Python community, and for *years* has been working on moving closer to the Python mainstream, lowering the fence for non-experts, mainly with the Zope 3 project. Significant work has been done to work with community standards like WSGI, and Zope 3 is, for instance, using Twisted. We’re moving on to embrace eggs now.

Whether the engagement of the Python community has succeeded fully can be debated, but clearly the awareness does exist and significant progress has been made. It’s a work in progress.

I don’t believe that criticisms of Zope have not been taken seriously, They might not have been taken seriously by some, but the community as a whole? Anyway, “just don’t get it” is of course not a good observation if you want to engage people as a community. I hope I haven’t been found to make such observations and I will try to discourage such attitudes if I am able to. Perhaps you can point me to one such recent interaction so I can keep an eye on this unfortunate tendency.

Please also note that the Zope community has a significant amount of existing codebase to move forward, and like everybody else, has limited developer volunteers, so even though criticism might be recognized, doing something about it often takes more time.

Finally, the web-track at the EuroPython conference in 2006 was organized by Paul Everitt and Godefroid Chapelle, both prominent Zope community members; we didn’t have a specific Zope track anymore. It was a clear recognition by Zope community members that the Python web landscape had changed, and the tracks were integrated this year as a response. We might’ve missed the boat on that in 2005, but before then, I am not sure whether a web track might’ve worked. That said, another factor is that quite a lot of the attention shifted to Plone, which has conferences of its own, so the EuroPython conference’s role changed.

So, Zope has a community of its own with meetings of its own. That’s the point I was trying to make in my original post – that it happens for Python-based web frameworks. And Zope has a problem integrating with the larger Python community – my point of this post is that yes, this is true, and that we aware of it, and that having a separate community is not necessarily a *bad* thing, but that we are actively working on improving its negative sides.

Anyway, you’re right, all this contributes to a credibility problem. It’s a difficult problem to address, as it seems to have resulted in significant Zope blindness in the wider Python community. I can see where our track record contributed to that, but on the other hand I believe a lot of value and honest effort by the Zope community is being ignored by the Python community because of this blindness.

I don’t think we have a significantly different view of the situation, Martijn, but your observations go some way to explaining why people don’t consider Zope as a Python Web technology in itself, but instead as a separate technology with its own community, customs and culture.

On the “don’t get it” remark, you certainly haven’t accused others of not getting it, but rewind in this thread and we have some kind of example:

“”"
# Willem de Gier Says:
August 23rd, 2006 at 6:23 pm

Django isn’t even close to where Zope was 8 years ago.
“”"

Zope eight years ago (August 1998) was certainly a usable application development framework, albeit with DTML at the forefront and with things like ZClasses as the upcoming state of the art. However, the sentiment of the remark is that of condescension – those Django upstarts have a lot to learn – whereas Django development involves a different set of tradeoffs that may be no worse than Zope in many cases, and Zope could quite possibly be strengthened to cater to those who prefer Django in such cases.

The eight years ago remark is interesting since it was more or less at that point that I found myself reading the Zope mailing list as much or more than comp.lang.python. Web programming back then really was a big new thing, and Zope was the way to do it: reading the Zope list and immersing oneself in the “zen” (talk about “not getting it”, eh?) was supposedly the way to become a master of the art, and if you wanted to learn about Web development, people on comp.lang.python often sent you to the Zope list to learn more.

So, perhaps Django is like Zope eight years ago in the sense that it will one day find itself isolated from Python and in that community’s blindspot. Perhaps the recent pronouncement is GvR’s way of saying that he’d rather have the new “chosen path” be an integral part of the Python community and that all the discussion should take place in all the same places that normal Python discussion happens. Or perhaps it was just an ill-advised recommendation of some technology that will eventually lead to nothing.

Greg,
On your criteria, maturity, customer value; I’m not buying this argument. I’m shocked, *shocked*, you didn’t mention Zope/Plone

Check out Sean Kelly’s screencasts:

http://seankelly.tv/blog/blogentry.2006-07-09.2857417511

and define “maturity” in web frameworks

If you get past the complexity (big if for many people) Zope/Plone blows away everything out there. Drupal would be coming in second. Actually Drupal could be coming in third, cos there’s also Zope/CPS (lesser known than Plone), which is pretty good.

I gotta say, I think the Python web frameworks world is in rude health, and I think Adrian/Martijn are on the money with their observations. I’ve never been happier

“So, perhaps Django is like Zope eight years ago in the sense that it will one day find itself isolated from Python and in that community’s blindspot.”

If so, the Django community will hopefully respond as the Zope community did years ago – try to move towards the Python mainstream where possible, and innovate where it makes the framework special. The Django community might learn, like the Zope community did, that this is a challenge. It’s starting to pay off for us though.

The Zope community is on the long road of making it back to perceived relevance in the Python community (I claim it’s already relevant, it’s just not perceived as such). I believe Zope’s focus on component-driven development along with improvements in the Python infrastructure that makes it easier to distribute such components will eventually have an impact.

Anyway, if this happens to Django, they’ll have the benefits of their own fresh insights along with the lessons of the Zope community, just like Zope is learning from them now. If at least Zope blindness won’t get in the way then.

just an update, Guido calls the idea of picking one web framework “a silly thing”. So while he clearly prefers Django, it looks like hes finally spoken definitively against the notion that “too many web frameworks is hurting python”…which is what I’ve wanted to hear all along. diversity, choices and competition are good things.

Sounds familiar, anyone? (shamelessly parodied from comp.lang.python, circa 2002)

Q: “I am a Python newbie, can you recommend a GUI framework for a quickie GUI app?”

A #1: “Tcl is best.”

A #2: “No, wxWindows.”

A #3: “PyGTK2 rocks.”

A #4: “PyQt rules, man!”

A #5: “There are > 5 (refs included), you should take a look around and judge for yourself.”

A #6: “It depends….”

Surely Mr/Mrs Newbie is well-qualified to judge Python GUI frameworks’ inherent qualities and has nothing better to do with his/her time. Say, what about using that old VB6 license after all?

While I appreciate choice, I feel _some_ guidance is useful too.

+1 to Ian Bicking’s comment about having a compelling story for new users.

Rails is mature and reliable? I can only guess you don’t talk to the staff at many Rails hosting companies.

Anyway, getting to address the two perceived issues:

* Sales of books = number of ignorant users
* Technical pub nights = number of users who don’t have lives or girlfriends

All I gather from this is that Pythonistas are smarter and more socially acceptable than their Ruby counterparts. Probably sexier too.

I can live with that.