Seriously, where is the source code?

Update: I sent this comment to the program committee of a conference as response to a recent review. It does not matter which conference; it goes for most of the conferences I am familiar with…

Dear program committee of conference X,

How is it that in this day and age you are still letting authors submit papers without disclosing their computational scripts? Most modern papers in our field rely heavily on computational methods and without being able to see the actual implementation it is impossible to assess whether the results are worth anything at all. Without seeing the actual code, “we used CVX” could mean just about anything; for example that the authors might not be solving the optimisation problem they think they are solving. I think it is downright frivolous to think that we can still get away with letting scientific research papers be superficial advertisement for the real scholarship that is tested in the computational code, but hidden away so that no-one has any chance of assessing the actual substance of the results being advertised. Further, asking reviewers to spend our time on it seemingly without considering this borders on being rude.

Advertisement

Inspiring Open Science Talk by Arvind Narayanan

I came across this talk by Arvind Narayanan a couple of days ago. He talks about how he has succeeded in publishing in unconventional ways and makes suggestions on how we might change the current publishing model – a topic that has become a bit of a hobby of mine lately:

http://33bits.org/2013/07/15/academic-publishing-as-ruinous-competition-is-there-a-way-out/