Date Submitted: Oct 8, 2019
Open Peer Review Period: Oct 8, 2019 - Dec 3, 2019
Background: Mapping out the research landscape around a project is often time consuming and difficult. Objective: The current study evaluates a commercial artificial intelligence (AI) search engine (IRIS.AI) for its applicability to automated literature search on a particular medical topic. Methods: To evaluate the AI Search engine in a standardized manner, the concept of a science hackathon (Scithon) was applied. Three groups of researchers were tasked with performing a literature search on a clearly defined scientific project. All participants had a high level of expertise for this specific field of research. Two groups were given access to the AI search engine IRIS.A. All groups had the same amount of time for their search and were instructed to document their result. Search results were summarized and ranked according to a predetermined scoring system. Results: Final scoring awarded 49 and 39 points out of 60 to the groups using AI, and 46 points to the control group. 20 scientific studies with high relevance were identified. Conclusions: AI technology is a promising approach to facilitate literature search and management of medical libraries. The scientific hackathon strategy might be an efficient tool to perform a literature search and to objectively compare different search strategies.