A history of enterprise search 2010 – 2017

by | Oct 17, 2017 | Search

From 2010 to 2013 there was a rapid consolidation in the enterprise search business. Between 2010 and 2012 Exalead was acquired by Dassault (2010), Autonomy by Hewlett Packard (2011), Endeca by Oracle (2011), Vivisimo by IBM (2012) and ISYS Search by Lexmark (2012). Some of these vanished without trace, some notionally exist (Exalead) and of course Autonomy has now re-emerged following its acquisition by Micro Focus. Others emerged to fill the gaps. Funnelback was initially developed by CSIRO in Australia in 2001 but did not really move into the limelight until the establishment of a UK office in 2009 following its acquisition by Squiz. Lucid Imagination was set up in 2009 and was then renamed LucidWorks in 2012. BAInsight dates back to 2003 as a supplier of add-on modules to SharePoint but over the last few years has repositioned itself as more of a systems integration company. Mindbreeze, an Austrian company offering a search appliance, was founded in 2005 but as with the other companies mentioned above has flourished over the last few years. Mention should also be made to Google, which launched the GSA appliance in 2002. In 2016 it announced it was leaving the market and licences terminate in 2018.

Looking back at the Gartner Magic Quadrant for Information Access Technology in 2005 there were four companies in the Leader/Visionary Quadrant, and they were FAST, Autonomy, Verity and Endeca. The majority of the companies surveyed in 2005 were towards the lower end of the Ability to Execute axis, and that has always been a challenge for the enterprise search business. Many companies with very good technology could not generate sales and cash flow to finance the marketing and sales effort needed to get to a critical mass. Over the last decade the market has been dominated by Microsoft SharePoint in terms of an installed base of search functionality (perhaps close to 300,000 installations?) though Google built up a substantial installed base of appliance servers before leaving the stage. The Enterprise Search Summit was launched in New York in 2008 and the exhibition space was full with around 40 vendors. Those were the days. The Enterprise Search Europe event was launched in 2011 but 2015 marked its closure as there were just not enough sponsors to keep the delegate fee at a sensible level. Thanks to Findwise we do now know much more about the way in which enterprise search is being implemented and used through the Enterprise Findability Surveys that started in 2011 and continued to 2016.

So what about the next decade? So much of the basic technology of search is several decades old but it works. To be sure there will be significant developments, as we have seen recently through the use of tensor maths in the Vespa relevance engine. Artificial intelligence and machine learning, together with continuing advances in natural language processing will also have a role to play. There are however two very significant barriers to adoption and growth. The first is that the search industry has totally failed to work together to educate the market to the benefits of search. Vendor web sites sell technology but business managers need to see increases in business, not technical, performance. IT may own the budget but only for the technology. All the search team support has to be provided by the business and vendors seem incapable to recognising the role of people in making search work.

The second barrier is that there is no training for search managers. Universities turn out people with a background in information retrieval but that is not enough to be a search manager. Even then it is not possible to get an undergraduate degree in information retrieval. In 2011/2012 (though it was not published until 2013) I undertook a study for the European Commission on the enterprise search market in the EU. The barriers to the adoption of enterprise search I identified as an outcome of four months of research remain in place today. Nothing has changed, except the rate of growth of the volume and value of information now being stored invisibly and inaccessibly inside organisations.

Martin White

(A history of enterprise search 2000-2009 17 October 2017)