How not to benchmark something

Jonathan Ellis from Cassandra wrote blog post "How not to benchmark Cassandra" (here: http://www.datastax.com/dev/blog/how-not-to-benchmark-cassandra-a-case-study) and presented us his own benchmark results for Cassandra, Couchbase, HBase and MongoDB. Ironically, his benchmark methodology is even more obscure than Thumbstack's one (read his post to understand the context). As usual, every vendor "tune" benchmark in his own favor and run competitors in default configurations, which is (at least for HBase) often is sub-optimal. We need an independent entity to conduct and publish NoSQL benchmarks.