An assessment of the search functionalities of one website

3. Written report Following your investigation of the selected search system, and engagement with literature, you must write up your findings into a written report that is worth 100% of the module mark. Sections 3 and 4 provide more details on the structure of the report. You should write your report for the website owner. They may have technical expertise and be familiar with the domain, but less familiar with search systems and evaluation. Your report should be structured as follows. A short description is provided about what to include in each section, the percentage of marks assigned for each section, and a guideline word count for each section.

3.1. Structure of report Structured abstract. Include a structured informative abstract (no more than 250 words – this is not included in the word count for the assignment) that contains the following sections: • Aim(s) of the report. • Selected website / search system. • Main findings (including strengths, weaknesses and improvements). No more than 250 words – this is not included in the word count for the assignment. The abstract carries no marks, but should be present. Marks will be deducted from “presentation” if you do not include it. Table of contents. Include each of the core sections identified here; use the numbered style, e.g., 1.0. 1.1, 1.2, etc., and include page numbers. The Table of Contents is not included in the word count for the assignment or the overall mark, but should be present. Marks will be deducted from “presentation” if you do not include the Table of Contents. Description of selected search system [15%]. You should briefly describe the purpose of the website, the users and likely uses. Where possible, you should provide (e.g. 2-3) references to support your description of the user’s, and their information needs and tasks. In this section you should describe the functionalities offered by the search system using the four categories proposed by Wilson (2012): input: features, control features, informational features and personalisable features. You should present this in the form of a table (not included in word count) stating category, feature provided and its purpose. You may use screenshots to aid your description where applicable.

 

This section will be assessed based on the following marking scheme: • 5% – the purpose of the website / search system, typical users and uses (e.g. information needs, search tasks, etc.). • 10% – list of the available search features to support users for input, control, informational features and personalisable features (Wilson, 2011). [Guideline: 450 words] Supporting the user’s search process [15%]. In this section you should describe the search tasks and queries created in 2.2. You can provide these in the form of a table (not included in word count) in an appendix which describes: • The type of search tasks you have chosen. • The search scenario(s) you have written for each type of search task. • The queries you have used for each search scenario. Based on your investigation of using the search system (2.3) to carry out your search tasks (2.2), you should walk through the search tasks and how the system responded, highlighting particular issues or useful features. You may find it easier to discuss the results of this study within the typical search process, e.g. using the retrieval model presented in (Sutcliffe & Ennis, 1998), and making use of screenshots where applicable. This section will be assessed based on the following marking scheme: • 5% – brief description of approach used and selection/description of the two search tasks (including scenarios and queries). • 10% – description of how the system has supported the user during the search process for the specific search tasks selected. [Guideline: 450 words] Analysis of retrieval performance [20%]. In this section you should briefly describe the purpose of this testing, the tests carried out, queries used, how you judged relevance and the measures of retrieval effectiveness used. Where possible, you should provide references to appropriate literature to support your testing (e.g. in selection of measures). You should provide a list of the queries used and summary of the score for each query in the form of a table (not included in word count) in an appendix which describes: • The queries selected (and any other information, such as category). • How relevance was judged. • Results for the measures used for each query. You should provide a brief summary of the results for this part of the assessment, highlighting the strengths and weaknesses of the retrieval system based on the results for particular queries, or groups of queries (e.g. for variants of a query). This section will be assessed based on the following marking scheme: • 10% – use and description of methodology (including queries, descriptions of relevance and measures used). • 10% – Description, analysis and explanation of results. [Guideline: 700 words] Analysis of usability [20%]. In this section you should briefly describe the testing carried out to assess usability. Where possible, you should provide references to appropriate literature to support your testing. You should provide a table (not included in the word count) that can be listed in the appendix that lists the 10 Nielsen heuristics and provides a brief description of whether the website/search system deals with the heuristic effectively. You should provide a brief summary of the findings of your usability assessment. [Guideline: 700 words]