The identification of organic colorants is of high importance in the cultural heritage field, where they are found as paint components and textile dyes, and in forensic science, because of their use in inks and paints, food colorants and textile dyes. Surface-enhanced Raman spectroscopy (SERS) has emerged as a promising technique for the detection of these materials, yet concerns over the sensitivity of SERS spectra of dyes to chemical and instrumental variables (such as pH, choice of SERS substrates and/or aggregants, and excitation wavelength) have prevented its widespread use in analytical applications. Over the last few years, the development of several microanalytical approaches has considerably increased the chances of success in the identification of minute amounts of dyes by SERS. However, the need for searchable databases is still largely to be fulfilled. In this work, we have assembled the core of a comprehensive library which contains 100 Raman and SERS reference spectra of natural and synthetic organic colorants. Experiments to classify 20 query SERS spectra of dyes from a variety of museum objects were conducted using principal component analysis (PCA) and the correlation coefficient (CC) algorithm. The effect of spectral transformations such as baseline correction, selecting a standard frequency range, normalization, smoothing, as well as carrying out the search on the second derivative of the spectra, was systematically evaluated. With this study we demonstrate that SERS spectra of organic colorants can be reliably matched against a well constructed spectral library regardless of the instrumentation and the colloids used, and of the pH conditions at which the measurements were carried out.