Maybe not being the fan that you guys are, made it easier for me to spot, and not think it was such a shitty cop out.
I actually thought the pay off was set up perfectly, as all the clues were there from the begining.

The only things that were a surprise were the son and wife.

I even worked out who he was, as soon as he said the body was never found!