Deficits in emotion perception (the ability to infer others' emotions accurately) can occur as a result of neurodegeneration. It remains unclear how different neurodegenerative diseases affect different forms of emotion perception. The present study compares performance on a dynamic tracking task of emotion perception (where participants track the changing valence of a film character’s emotions) with performance on an emotion category labeling task (where participants label specific emotions portrayed by film characters) across seven diagnostic groups (N = 178) including Alzheimer’s disease (AD), behavioral variant frontotemporal dementia (bvFTD), semantic variant primary progressive aphasia (svPPA), non-fluent variant primary progressive aphasia (nfvPPA), progressive supranuclear palsy (PSP), corticobasal syndrome and healthy controls. Consistent with hypotheses, compared to controls, the bvFTD group was impaired on both tasks. The svPPA group was impaired on the emotion labeling task, whereas the nfvPPA, PSP and AD groups were impaired on the dynamic tracking task. Smaller volumes in bilateral frontal and left insular regions were associated with worse labeling, whereas smaller volumes in bilateral medial frontal, temporal and right insular regions were associated with worse tracking. Findings suggest labeling and tracking facets of emotion perception are differentially affected across neurodegenerative diseases due to their unique neuroanatomical correlates.