Is enterprise search useful at all? Outcomes of a case study
I’m in the final stages of writing the 2nd Edition of my book Enterprise Search for publication in April 2015. I have spent an enjoyable day making sure that I am aware of any research literature on the subjects covered in my book. At the recent JBoye Aarhus conference I was a member of a plenary expert panel and used my slot to highlight the fact that very few practitioners look at the research literature. The reaction from my fellow panellists was that there was little of value in the research literature that was relevant to practitioners and that seemed to be the sense of the audience. Perhaps because of my background as a chemist I find the research literature fascinating and full of valuable insights.
As I was working through the ACM Digital Library my eye was caught by a paper entitled Is Enterprise Search Useful At All? Lessons Learned From Studying User Behaviour by Alexander Stocker and five co-authors from the Virtual Vehicle Research Centre, Graz University of Technology and Know-Centre Austria. There are in fact very few papers published on enterprise search, let alone on a case study. This paper reports on a group of ten engineers and how they apply enterprise search to their projects, how satisfied they are with the search application and what they perceive to be the overall value of enterprise search. The initial section of the paper is a quite extensive literature review (though it omits my book) of 41 references, a good indication of how little research has been published on enterprise search in practice. The engineers used SharePoint 2013 Foundation in an out-of-the-box configuration.
Among the issues that emerged (not surprisingly) were the difficulty of query formulation, inconsistent documentation and use of metadata, the heterogeneity of document content and structure and the misleading built-in relevance model of the search application. What also emerged was the need for help in developing the optimum search strategy. The project was very small scale, and the outcomes are mainly some interesting comments made by the engineers as they undertook searches. For those of us who are familiar with the vagaries and challenges of enterprise search there is nothing especially surprising about the outcomes but the paper is useful in putting enterprise search into context and providing a basis for further research projects along the same lines. The bibliography alerted me to some papers that I was not aware of, though only 6 of the 41 references date from 2011 onwards. Getting funding for research projects is much easier when there is an established methodology that can then be extended to other disciplines and circumstances. The paper was presented at the i-KNOW conference, held in Graz in September this year.
The downside of much of the research literature is that it sits behind a subscription firewall, and this paper is no exception. As you will see from the paper link above the conference proceedings are published by the Association for Computing Machinery and at present there does not seem to be a parallel publication from any of the three participating organisations.