This mysterious phrase if first said by Granny in Episode 1 of Series 1. Granny calls Punch her husband, but Punch isn't her husband as she's his mum and he's married to Judy. What's going on here??

A similar phrase 'It's the evil lady who married my Punch', is said by Granny to Judy in episode 2. It is later shown that Granny stronly objected to the marriage telling Punch not to marry 'that evil lady'.