Search Stress Test
At a search-by-search and day-to-day level you may well have only limited evidence as to how well your search applications are working. It is very easy to become complacent, especially if there is a work-around in asking co-located and remote colleagues if they have the information you are seeking.
For this reason I have evolved this set of twelve search stress tests (based on failed searches in client engagements) that will give you some sense of whether you can trust your enterprise (and/or intranet search) to meet your personal requirements. All of the tests are based on common enterprise enquiries and often feature in ‘moan-lists’ about search not working. Each test should only take 2 minutes to complete. If it takes you longer then you have failed the stress test even if eventually you track down the information. All twelve tests should not take more than around 30 minutes in total but of course there is no reason to do them in a single session.
You can either complete the Stress Test yourself or ask a few colleagues to undertake the test and share their results.
A good search application supported by a search team should deliver satisfactory results for all these tests. It is not just about the technology; these stress tests also check out the quality of the information management in your organisation.
There is no charge for the use of this Stress Test and there is no obligation to share the results with Intranet Focus Ltd.
Test 1 Specific document
Think of a document you wrote in 2018 that is stored as a pdf file. Then search for it without using any of the words in the document title. The reason for this is that probably only you can remember exactly what the title was, and it might be along the lines of ‘Q4 report on Project Solon’. You can use as many query terms as you like. The idea that people only use single query terms is erroneous. The choice of a pdf is to assess the performance of the indexing of pdf documents which present significant challenges.
Test 2 Version discovery
Select a policy or procedure document that has been through a number of (perhaps annual) revisions and then search for all the previous versions, justified by wanting to know when a particular section was added or deleted.
Test 3 Confidentiality
Search for the term [confidential] and reassure yourself that you do have permission to see all the results returned. It should also bring back the protective marking scheme for the organisation.
Test 4 Expertise discoverability
You do of course have some specific expertise that justifies why your organisation hired you. Using no more than two query terms see how high up you come in the list of potentially relevant experts. No-one is going to look beyond p1! The query term number is limited to 2 because in expertise search people often do not have the expertise to define the expertise they are seeking. Strange but true. If you do not appear on page 1 of the results (and so are potentially invisible) how many query terms does it take for you to be positioned on page 1? Then consider whether anyone will in fact use all these terms to track you down.
Test 5 Project files
Think of any project, either by name or objective, and find the definitive (signed off) final report of the project.
Test 6 Data search
Almost certainly you will be adding information into an Excel spreadsheet on a regular basis. Can you find the last version published in FY2019 (not calendar year) just to check that 2019 information has been correctly included in a document. Not the latest – that is too easy.
Test 7 Presentation retrieval
Recall a PowerPoint presentation you gave in 2019 and search for it using query terms taken from the agenda slide. Does it turn up in the top 10 results?
Test 8 Given name search
The range of family names is always greater than given names. Imagine that someone in a Zoom/Teams meeting suggests that ‘John is the person to talk to’. The assumption is that you know which John (or is it Jon or Jonathan?) and what their role and (virtual) location is. Search for a colleague just using their first name and see how long it takes you to track down their profile, role and location.
Test 9 Phonetic name search
You will almost certainly have a colleague whose given name and/or their family name is not spelled the way that it is pronounced. Search for their profile based only on how their name sounds, not how it is spelled.
Test 10 Your visibility
Use your name to find out what information about you and your work is immediately accessible to other employees. Is it accurate, representative and current?
Test 11 Customer contact
Imagine that you have come across information about a customer or a supplier that you think could be of value to your organisation in enhancing the way in which the customer or supplier sees a benefit in working with you and your colleagues. Can you find out who is the lead person in managing the relationship with the customer or supplier?
Test 12 Application support
Select a corporate IT application that you rarely (if at all) use and search for information about how to use the application, including on-line training and comments on social media regarding tips, tricks and work-arounds. Do you feel you have enough to open up the application and use it successfully?