Something blue

I recently met an old friend of mine who I was surprised to find out was still harbouring a grudge against me or more specifically my actions.
Turns out about 10 years ago he got married and he is still upset at the fact that I drew a cock and balls in his wedding book.
Now before you decide to cast your judgement, hear me out.

This friend, who I have not seen in years, said he was still upset about it and his wife went BALLISTIC over it. I apologised for any hurt or upset caused as that was clearly never my intention but also tried to point out that it was just a bit of fun. He then hit back with "How would you feel if I drew a cock and balls in your wedding book?"
My response was "honestly, I think I'd take it in the spirit it was intended."
He then said that he is now ok with it as he knew why I had done this.
He went on to tell me that he understood that my partner at the time had a real dislike for him him and thus explained my actions.
At this point I felt compelled to defend myself.

I did not draw the boaby for any other reason than I was drunk.
The drawing of a boaby is a juvenile act that I still find funny even at my age and will no doubt continue to do as always when presented with the opportunity, whether that be in a hotel comments book, a suggestion box anywhere, on a voting slip, on a political street filled petition, when paying funds into the bank (where the signature is not important), when signing for an delivery or infact when drunk and handed a wedding book.

I do sincerely apologise if my actions detracted in anyway from the day, book or marriage as a whole as that was never my intention however to think that it was done through malicious eyes is nothing short of slanderous and as upsetting to me as the offending doodle seems to have been to the bride. It's not like I wrote that either the bride or groom was a dick. It's just a funny drawing of a boaby.
I think they're more annoyed that they couldn't just rip the page out as the Granny had written her message on the other side of the page. (Something or other about a fantastic boaby no doubt.)
Truth be told, I believe the person directly after me did try and turn it into a love heart but only made things worse.

Anyway, can I have your thoughts on this, the latest in a long line of scenarios I find myself in where I am apologising for actions that I don't feel merit an apology.
I think if you do something for a laugh then have to explain why you did it, it detracts from the initial humour an infact renders the whole thing NOT actually funny.

What do you think?
Are they oversensitive or am I insensitive?