Abstract: We created a visualization tool that helps Automatic Chord Recognition (ACR) developers to characterize system performance across a test data set. Our system's design uses Information Visualization (InfoVis) principles to communicate accuracy more effectively than a table of mean metric scores. We share some of the insights we developed while building our tool, and hope our findings may help inform the design of figures used in future publications, and affect how future ACR system designers improve and present their systems.