Librarians often customize discovery system settings based on instincts or on small usability studies. A/B testing, commonly used by commercial enterprises but infrequently by libraries, employs two simultaneous, live versions of a web interface to gauge the effects of changing variables. This method has the advantage of reaching a large number of users performing authentic search tasks. The presenters applied this technique to their discovery system, Primo, by automatically redirecting search traffic to one of two versions. Variables such as facet labels, facet order, and placement of search refinement options were sequentially tested and the resulting user data analyzed for effects on behavior. The results provide insight into the extent to which interface changes prompt users to employ available search refinement options, allowing libraries to make customization decisions driven by user data. Effects of discovery system configuration on the quality of information selected by students will also be discussed.