As emergency department (ED) crowding continues to worsen, many visits are at academic referral hospitals. As a result, engaging specialty services will be essential to decompressing the ED. To do this, it will be important to understand which specialties to focus interventions on for the greatest impact. To characterize the ED utilization of non-surgical adult patients with an ambulatory specialist who were seen and discharged from the ED. Retrospective cohort study of all consecutive patients currently under the care from a specialist presenting to an urban, university affiliated hospital between 01 January 2015 and 31 December 2016. The identification of ED visits attributable to specialists was based on the primary diagnosis of ED visits and the frequency of visit with specialists within a given timeframe. Only patients who were discharged directly from the ED were included in the analysis. There were 29,853 ED visits by patients currently under the care of a specialist during the study period. 17.76% of these visits were related to the medical specialty of the specialist. Of these visits, 41.73% occurred during office hours, and 24.81% occurred during weekends. The specialties with the largest proportion of ED visits related to their specialty was cardiology, gastroenterology, and pulmonary, respectively. Nearly 18% of all patients that have a specialist and are treated and discharged from the ED present with a diagnosis related to their specialist’s practice. This may indicate that there is a role for specialty service to play in decreasing some ED utilization that may be appropriate for the out-patient clinical setting. By focusing attention on specific specialties and interventions targeted during office hours, there may be an opportunity to decrease ED utilization.