Turning search upside down : search for queries with documents

05/26/2014 - 14:10 to 14:50
long talk (40 min)

Session abstract: 

While normal search engines allow you to search over large document collections with individual queries, monitoring systems have different requirements: to run large numbers of stored queries over a stream of incoming documents, and provide exact matches for those queries that hit.  This can be an expensive process, in terms of time, memory and hardware.  In this talk I'll discuss the luwak open-source library, built by Flax to solve just this problem, and show how we reduce the problem space by turning the problem upside down, converting queries into documents, and documents into queries.