detriment

ˈdɛtrɪm(ə)nt

noun

the state of being harmed or damaged.

he is engrossed in his work to the detriment of his married life