Hey, Faraway Lad, I grew up in the US and I didn't "get" it until a few years ago either. I think I equated "bugger" (as a noun, like 'that little bugger') with BUG as in PEST. And for something like "Well I'll be buggered" I just took it for an exclamation of surprise. (Which it is, but still.) Interesting what words can convey even when the exact/literal meaning(s) in unknown or idiosyncratically interpreted.

Next up in linguistics class: "over a barrel"