One of my linguistics professors (with a thorough knowledge of Germanic historial phonology) used to do this. He told us how, at one point, his hosts politely asked him to please stop and speak English instead!