CommitFests and Meritocracy

Yesterday, I wrote a blog post on whether and to what extend the PostgreSQL community is a welcoming community, in which I quoted some remarks that Selena Deckelmann made, and she responded with her own post on where meritocracy fails.  I want to just recap a few things that may not be totally obvious to casual observers of our community; and then I have a few remarks about meritocracy.

A Welcoming Community?

I attended PostgreSQL East last week and, as usual, the most interesting discussions were the ones that nobody planned.  Ed Boyajian, the CEO here at EnterpriseDB where I work, threw out a remark to the effect that the PostgreSQL community is a meritocracy.  Selena Deckelmann wasn't convinced, "not to say that we don't do try hard, and do pretty well."  During the closing session, Josh Drake made some remarks to the effect that, in effect, the process for submitting a patch to PostgreSQL is long, difficult, and fraught with arbitrary and capricious rejection, which caused me to rise to my feet and object that that while it's not perfect, I think we do pretty well, and better than we used to.  Josh agreed with some of what I have to say, and I see his point, too.  But it got me thinking about the whole way that we operate as a development community, and what is good and bad about it.

Working Toward PostgreSQL 9.1beta

I'm pleased to report that we seem to be making good progress toward PostgreSQL 9.1beta.  As we did for PostgreSQL 9.0, we are maintaining a list of open items that we need either to address or to decide that they aren't so important after all (there are also some issues that are being discussed on the mailing list that aren't reflected in that open items list).  What we're really doing here is trying to stabilize the code, particularly with respect to the features committed at the very end of the development cycle.  So far, the main problem areas appear to be (1) collation support, (2) changes to streaming replication, especially the new synchronous replication feature, and (3) the new serializable isolation level.  But we are making rapid progress in sorting out the loose ends, and I feel a lot better about the release now than I did even a week ago.

Barring objections or unforeseen problems, I'm hoping to bundle an alpha5 release a week from today, on Monday, March 28.  We haven't discussed the time line for beta yet, other than to hope that we'll be able to get there in April.  I still think that's a realistic timeline, although there is quite a bit of work that must be done between now and then for us to hit it.  In the meantime, if you're in a position to build from source, please keep testing and sending in bug reports.  Thanks!

Query Planner Enhancements in PostgreSQL 9.1

PostgreSQL has an awesome query planner.  And it keeps getting better.  postgresql.org has a survey asking people Which PostgreSQL 9.1 Feature Are You Most Excited About? (to vote on it, if it's still open when you read this, go here).  For some reason, query planner features tend not to be listed when people talk about what they're most excited about in a new release, but there's ample room for excitement.  If a particular query planner feature happens to apply to your use case, it's possible to see multiple-order-of-magnitude speedups, avoiding the need for a major application redesign or even an expensive hardware purchase.  This is a big deal.

Advice for Google Summer of Code Students (and other prospective contributors)

Periodically, someone whose name I've ever seen before contacts me -- or posts to pgsql-hackers -- to say that either (1) they've already written a great patch for PostgreSQL and they'd like to know how to get it committed or (2) they're interested in writing a patch to implement some major new feature in PostgreSQL.  Some of these people are prospective Google Summer of Code students, while others are researchers or other people who, for whatever reason, are interested in PostgreSQL.  I am always thrilled to see more people take an interest in PostgreSQL, but unfortunately I've seen a number of people who seemed very smart and promising crash and burn when in terms of actually making a successful contribution of code.  In fact, no matter how promising things seem at the outset, the failure rate - in my experience - is very close to 100%.

More Musings on Logical Replication

There were some interesting comments on my previous blog post on logical replication, which have inspired me to write a bit more about this topic.  Exactly how should this actually work?  Every time a tuple is inserted, updated, or deleted, we already write to the transaction log the entire contents of any new tuple, and the physical position of any old tuple.  It would be nice to piggyback somehow on the work that's already being done there, but it's hard to see exactly how.  I see three main problems.

Troubleshooting Stuck VACUUMs

Over the years, I've occasionally encountered situations where VACUUM fails to make progress, and not fully understood why that was happening.  Recently, I've come to a better understanding of how lock conflicts can result in VACUUM stalling out either at the beginning of the table, or part-way through.  (If you're not already familiar with the types of locks that PostgreSQL uses, you may find it helpful to read through my earlier blog post on locking in PostgreSQL before reading on.)