In The Scarlet Letter, why does Chillingworth think he has a double reason for punishing Dimmesdale?

In The Scarlet Letter, why does Chillingworth think he has a double reason for punishing Dimmesdale?