DFIR Training Blog



If Peer Review is so Important, Why Doesn't Everyone Do it?

I have posted on this a few times, as well as commented on Twitter, but the short answer is: "We don't peer review because it is too much work and too much time spent with no real personal benefit."  Our jobs are not publishing, but actually practicing the trade of DFIR. 

Now I see another reason why DFIR researchers may not be publishing their work via the 'academic journals'. 

I feel that DFIR had been doing it right all along. Practitioners work. They find something interesting. They blog about it. Then everyone else takes advantage of their discovery. And when it's really good, the practitioner writes up a Word doc, PDFs it, and uploads to the Internet. Now it is memorialized forever (or until the Internet dies). I had suggested that the DFIR community add one, little step between the PDFing and posting: community peer review. The reasons to add one thin layer of peer review is simply to validate the work that was done so that citing it becomes easier and the DFIR discoverer gets permanent credit.  The community benefits overall.

After reading " Some science journals that claim to peer review papers do not do so ", I see that there are even more reasons to avoid the academic route to journals unless your job is in academics or you want to go into that field.

  • Pay for play? Come on now.
  • Community access? Nope. Not unless you pay, or have insider access like if you worked at a university.
  • Fake journals? Yep. Apparently so.

I agree that there is validation, credibility, and personal satisfaction in having an academic peer-review paper that is published in a journal, but everything that is required to do so goes against the very grain of DFIR work. DFIR research needs to be shared yesterday, not two years from tomorrow. The methods and artifacts that we discover are sometimes perishable, but certainly they are dynamic. The academic model for peer review doesn't work for DFIR research because it takes too long.  In fact....very few practitioners read the scientific journals, and with that, the research will have been in vain.

My bias* as a practitioner is obvious, because there is no hurry in the academic world. The academic world does not deal with a breach where a business may go bankrupt in days, or where national security secrets are being siphoned out of a network, or where a child needs to be rescued after being lured online. Practitioners need the newest research as soon as it is ready ( ready to be put to use, not ready for the academic peer review process ). 

As a matter of practicality, money probably needs to be involved in this process, because although I support working a job for fun, I do not agree that you should be required to work for free. How a business model needs to be developed for a non-academic peer review model is a topic that should be started sooner rather than later. The good news is that I see more than a few DFIRrs talking about it.  Now that is cool.


*Side notes on my perspective and bias:

I have practiced DFIR in the public sector and private sector, and taught it in the academic world. I tend to see the importance of immediate access to research being more overriding in importance than a long-process of publishing.

Written by :Brett Shavers

{rscomments option="com_rsblog" id="29"}