The potential to provide inferences about fish abundance from environmental (e)DNA samples has generated great interest. However, the accuracy of these abundance estimates is often low and variable across species and space. A plausible refinement is the use of common aquatic habitat monitoring data to account for attributes that influence eDNA dynamics. We therefore evaluated the relationships between eDNA concentration and abundance of bull trout (Salvelinus confluentus), westslope cutthroat trout (Oncorhynchus clarkii lewisi) and rainbow trout (Oncorhynchus mykiss) at 42 stream sites in the Intermountain West (USA and Canada) and tested whether accounting for site-specific habitat attributes improved the accuracy of fish abundance estimates. eDNA concentrations were positively associated with fish abundance, but these relationships varied by species and site, and there was still considerable variation unaccounted for. Random site-level differences explained much of this variation, but specific habitat attributes of those sites explained relatively small amounts of this variation. Our results underscore that either eDNA sampling or environmental characterization will require further refinement before eDNA can be used reliably to estimate fish abundance in streams.