Appendix A – tasks Note: all URLs and rankings were live and correct at the time of evaluation, but may have changed since the evaluation ended. COHSE-NeLI evaluation tasks =========================== Short format (4 tasks) ---------------------- 1. What percentage of viral encephalitis and meningitis cases in the UK are of undetermined aetiology? COHSE Answer: (in “Results” section of target document): “In the UK up to 60% of viral encephalitis and meningitis cases are of undetermined aetiology.” Target URL: (first link in link box): http://www.hpa.org.uk/web/HPAwebFile/HPAweb_C/1194947364642 Title: “Enhanced surveillance for UK-acquired West Nile Virus 2002-2005” Search terms: “viral encephalitis” NeLI alone Example of partial answer: “Herpes simplex is the commonest cause of sporadic viral encephalitis. However in many typical cases no aetiological agent is identified.” URL: http://www.hpa-standardmethods.org.uk/documents/qsop/pdf/qsop48.pdf Title: “Investigation of Viral Encephalitis and Meningitis (QSOP 48) Search terms: “viral encephalitis” Rank: 2nd Using COHSE, a click on the highlighted term in the title of the document containing the partial answer would retrieve the one containing the full example. 2. Were there any travel restrictions to Turkey at the time when it was affected by avian influenza? In what areas of Turkey did Europe’s first confirmed human cases of avian influenza occur? COHSE Answer: (in third paragraph of target document) Dogubayazit Target URL: http://www.nathnac.org/pro/clinical_updates/anian_influenza_Turkey_090106.htm Title: “Additional human cases of avian influenza (H5N1) confirmed in Turkey” Search terms: “avian influenza” NeLI alone The first part of the question “Were there any travel restrictions to Turkey at the time when it was affected by avian influenza?“ could be answered as follows: Example of partial answer: (in paragraph “Advice to travellers”): “WHO does not at present recommend any restrictions on travel to any country currently experiencing outbreaks of bird flu in poultry flocks, including countries which have also reported cases in humans.” URL: http://www.dh.gov.uk/en/Aboutus/MinistersandDepartmentLeaders/ChiefMedicalOfficer/Features/DH_4102997 Title: “Bird flu and pandemic influenza – what are the risks?” Search terms: “avian influenza” Rank: 3nd Example of partial answer: “Currently there are no travel restrictions to countries reporting avian influenza.” URL: http://www.nathnac.org/travel/news/avianinfluenza_traveladvice_070406.htm Title: “Avian Influenza – advice for travellers to affected areas” Search terms: “avian influenza” Rank: 12th Example of partial answer: “WHO does not at present recommend any restrictions on travel to any country currently experiencing outbreaks of H5N1 avian infection in poultry flocks, including countries which have also reported cases in humans.” URL: http://www.who.int/csr/disease/avian_influenza/travel_2004_02_11/en/ Title: “Avian Influenza – advice to international travellers” Search terms: “avian influenza” Rank: 13th 3. What is an example of a chemical that can cause or exacerbate acne? COHSE Answer: “The principal adverse health effect of exposure to dioxins is a form of acne known as chloracne. This is a severe skin disease which mainly affects the face and upper body with acne-like spots that may be present many years after exposure.” Target URL: http://www.hpa.org.uk/web/HPAweb&Page&HPAwebAutoListName/Page/1190384322195 Title: “Additional human cases of avian influenza (H5N1) confirmed in Turkey” Search terms: “acne” NeLI alone Example of partial answer: (in section “What Can Make Acne Worse?”) : “Certain medicines and chemicals.” URL: http://www.netdoctor.co.uk/diseases/facts/spots.htm Title: “General Information About Acne” Search terms: “acne” Rank: 6th 4. What kind of certificate should be used for documenting yellow fever vaccination? Have there been any changes to the format in the past two years? COHSE Answer: the new style International Certificate of Vaccination or Prophylaxis, and the old Port 37-style went out on 15 December 2007 Target URL: http://www.nathnac.org/pro/news/YFcert121207.htm Title: “12 December 2007 - Validity of International Certificate of Vaccination or Prophylaxis – reminder” Search terms: “prophylaxis” or “vaccination” or “immunisation” NeLI alone Example of partial answer: (in last chapter on Yellow Fever, listed on this page): “Certain medicines and chemicals.” URL: http://www.dh.gov.uk/en/Publichealth/Healthprotection/Immunisation/Greenbook/DH_4097254 Title: “Green Book” table of contents Search terms: “vaccination” Rank: 1st Long Format (10 tasks) ---------------------- The long COHSE evaluation consisted of the above four questions, plus the following six. 5. Has the risk of dengue virus in Cambodia increased or decreased over the last two years? COHSE Answer: Cambodia had its biggest ever epidemic of dengue in 2007 Target URL: http://www.nathnac.org/travel/news/dengue_chik_180108.htm Title: “Dengue fever and chikungunya virus activity” Search terms: “dengue” or “dengue fever” NeLI alone Example of partial answer: In section “South-East Asia”, paragraph “Vector-borne infections” document states that “Dengue fever is hyperendemic in the region” and “List of Countries, Southeast Asia” Cambodia is listed URL: http://wwwn.cdc.gov/travel/yellowBookCh3-Asia.aspx Title: “Travelers Health – Yellow Book”, Chapter 3 “Geographic Distribution of Potential Health Hazards to Travelers” Search terms: “dengue fever” or “dengue” Rank: “Yellow Book” ranks 3rd and link goes to table of contents 6. Are there any infections that pose more of a threat in Latvia than they do in the UK? If so, one would not expect malaria to be among them, but could this expectation be wrong? A search on the term “Latvia” returns six documents: (yes – community-acquired MRSA, keyword search ‘Latvia’) COHSE Answer: 4th-ranked document (see below) searched for keyword “Latvia” leads to the overview of risks to travelers, and document contains section “Malaria” which states “CA-MRSA strains have also now been detected in France, Switzerland, Germany, Greece, the Nordic countries, Australasia, The Netherlands, and Latvia. In the United Kingdom, cases of CA-MRSA and PVL producing strains have been reported, but the number of reported cases is still small.” Target URL reachable only through COHSE states that there is NO risk of malaria in Latvia. Target URL: http://www.nathnac.org/ds/c_pages/country_page_LV.htm Title: “Country Information: Latvia” Search terms: “Latvia” NeLI alone Example of partial answer: community-acquired MRSA is more common in Latvia (see above). URL: http://www.neli.org.uk/IntegratedCRD.nsf/e67b3914fbe8da658025755c0062cd62/A358A33879F1937580257324003EDC13/$File/PIIS0196655307005950.pdf Title: “The effectiveness of hand hygiene procedures, including handwashing and alcohol-based hand sanitizers, in reducing the risks of infections in home and community settings” Search terms: “Latvia” Rank: 4th 7. What are three chemicals that have successfully been used for decontamination of areas affected by anthrax? COHSE Answer: hypochlorite solution (smaller areas); chlorine dioxide and formaldehyde (larger areas) Target URL: http://www.hpa.org.uk/web/HPAweb&HPAwebStandard/HPAweb_C/1204186180817 Title: “Environmental Decontamination” NeLI alone Example of partial answer: hypochlorite solution (smaller areas); chlorine dioxide (larger areas) URL: http://www.cdc.gov/ncidod/eid/vol11no01/04-0635.htm via http://www.bt.cdc.gov/agent/anthrax/index.asp Title: “HEPA/Vaccine Plan for Indoor Anthrax Remediation”, link in “Additional Research & References” section of “Anthrax: Environment/Response” Search terms: “Anthrax” Rank: 11th 8. How much risk of skin damage can the Mediterranean sun present to a typical redhead? COHSE Answer: “Notes” section states “The solar index does not exceed 8 in the UK (8 is rare; 7 may occur on exceptional days, mostly in the two weeks around the summer solstice). Indices of 9 and 10 are common in the Mediterranean area.” Above this is the section “Risk of Damage to Skin” which shows a table categorising the risk as “very high” at solar indices 9 and 10 for “Category 1”. “Category 1” is defined in the section “Assumptions” as “most UK indigenous groups” Target URL: http://www.hpa.org.uk/web/HPAweb&HPAwebStandard/HPAweb_C/1195733848114 Title: “Solar Index and Skin Type” Search terms: “skin” NeLI alone Example of partial answer: no hard figures in answer, but section “Especially Vulnerable Groups” lists “Fair skinned people who often also have red hair or blue eyes”, but without further detail URL: http://www.fitfortravel.nhs.uk/advice/advice-for-travellers/sun-exposure.aspx via http://www.fitfortravel.nhs.uk/advice/advice-for-travellers/sun-exposure.aspx Title: “‘fitfortravel’ (Scotland)” Search terms: “sun” Rank: 2nd 9. Which particular diseases are the focus of infection prevention efforts in prisons? COHSE Answer: Target document prominently displays the following list under “Related Topics”: • Blood borne viruses (in general) • Hepatitis A • Hepatitis B • Hepatitis C • HIV and Sexually Transmitted Infections • Injecting drug users • Meningitis C • Tuberculosis Target URL: http://www.hpa.org.uk/webw/HPAweb&Page&HPAwebAutoListName/Page/1191942126463?p=1191942126463 Title: “Prison Infection Prevention Team” Search terms: “Prison” NeLI alone Example of partial answer: document links include tuberculosis, HIV, hepatitis (including hepatitis C) URL: No single document, search on “Prisoners” returns 10 documents Title: No single document Search terms: “prison” or “prisons” or “prisoners” Rank: No single document 10. If a patient presented with uncomplicated dyspepsia that did not respond to lifestyle changes, what commonly used non-invasive test would be the most likely to give you a false positive for helicobacter pylori? Answer: lab-based serology (most common, least precise), breath test, and (monoclonal) stool antigen test (most precise) Target URL: http://www.hpa.org.uk/web/HPAwebFile/HPAweb_C/1194947387361 Title: “Test & Treat Helicobacter Management of Dyspepsia: Cost Comparison of Serology to Stool Antigen & Breath Test” Search terms: “dyspepsia” NeLI alone Example of partial answer: document links include tuberculosis, HIV, hepatitis (including hepatitis C) URL: No single document, search on “helicobacter pylori” returns 18 documents, of which the first leads to The Helicobacter Foundation at http://www.helico.com/ Title: No single document Search terms: “helicobacter pylori” Rank: No single document The CORESE-Based SWB evaluation tasks ===================================== 1. What are three disease prevention efforts in the USA to prevent and control the spread of viral haemorrhagic fever through rodent vectors? Answer: “Because many of the hosts that carry hemorrhagic fever viruses are rodents, disease prevention efforts include • controlling rodent populations; • discouraging rodents from entering or living in homes or workplaces; • encouraging safe cleanup of rodent nests and droppings.” Found in paragraph “How can cases of viral hemorrhagic fever be prevented and controlled?” Target URL: http://www.cdc.gov/ncidod/dvrd/spb/mnpages/dispages/vhf.htm Title: “Factsheet on Viral Hemorrhagic Fevers” Search terms: “viral haemorrhagic fever” Ranking on NeLI keyword search: not found 2. In the Netherlands in 2007, there was a cluster of infections in a care home caused by poor infection control practices. What was the infection and what (specifically) was its most likely cause? Answer: The infection was Hepatitis B, and “one case of chronic hepatitis B was found that could be identified by genotyping as the source patient for the acute cases. Most likely the cause of transmission was a multiclix finger stick device developed for use in individual patients but used in multiple patients.”) Target URL: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=18918 Title: “A cluster of hepatitis B infections associated with incorrect use of a capillary blood sampling device in a nursing home in the Netherlands, 2007” Search terms: “care homes” Ranking on NeLI keyword search: Below 20, though ranks within top 20 using the search terms “hepatitis B” which would only be used by someone who already knew or suspected the answer. 3. What are the recommended guidelines for hygienic cleaning of surfaces after flooding? Answer: On page 7 several paragraphs address hygienic cleaning of surfaces. Target URL: http://www.ifh-homehygiene.org/2003/2downloadabledoc/floods105.pdf Title: “COPING WITH FLOODS, CLEANING UP AFTERWARDS – AND STAYING HEALTHY - Advice and Information Sheet issued by the International Scientific Forum on Home Hygiene - July 2007” Search terms: “cleaning” Ranking on NeLI keyword search: Not found using numerous keyword and category searches, including “cleaning” 4. Which are the four main types of hospital unit within which outbreaks of VRE have been reported? Answer: In paragraph “what diseases do VRE cause?” it is stated “Outbreaks within hospitals of VRE infection have been reported mainly from renal dialysis, transplant, haematology and intensive care units.” Target URL: http://www.amm.co.uk/newamm/files/factsabout/fa_vre.htm Title: “The Facts About Vancomycin-resistant enterococci (VRE)“ Search terms: „VRE“, „Vancomycin resistant enterococci“ Ranking on NeLI keyword search: Not found using various keyword and category searches, including „VRE“ and „Vancomycin resistant enterococci“ GoPubMed/GoGene evaluation tasks ================================ None of the GoPubMed/GoGene tasks have specific target documents, because PubMed results change so frequently. Short format (4 tasks, PubMed vs GoPubMed) ------------------------------------------ Answered using PubMed 1. Which particular diseases are associated most often with HIV? 2. What is the role of MMS2 in cancer? Answered using GoPubMed 3. How does BARD1 regulate BRCA1 activity? 4. How rare/prevalent is Paget's disease? Long format (11 tasks, PubMed vs GoGene and PubMed vs extended GoPubMed) ------------------------------------------------------------------------ Answered using PubMed 1. What is the main role of the gene MMS2? Name 3 genes related to it in literature. 2. How are MMS2 and UBC13 related to each other? Answered using GoGene 3. Name 4 organisms that have an MMS2 gene. 4. How do mutations in "Sonic Hedgehog" genes affect developmental disorders? 5. What other genes are related to Shh in literature? Name 3 of them. 6. Are there any GO terms manually assigned to Shh? Name 3 of them. Answered using PubMed 7. Which biological process is Rab5 involved in? 8. Can you find any conserved domain information on Rab5? Answered using the extended GoPubMed 9. Which biological process is Apc11 involved in? 10. Can you find any conserved domain information on Apc11? 11. Which pathway are the zebrafish genes Her1, Her7 and DeltaC involved in?