During the Wagatha Christie libel trial at the High Court between Vardy and Coleen Rooney, Ms Vardy was asked about her tell-all interview in which she claimed she had a fling with the former pop star in 2001.
Mr Andre, 49, is upset that the claims originally made in the News Of The World in 2004 have been dredged up again and that he was given no warning they would be detailed in Ms Vardy’s court battle.
“If a man had said this about her anatomy and made up something -you can use your imagination, saying something very unflattering – there would be outrage. There would be absolute outrage,” Mr Andre said in his first response to the remarks.
“Because it has been said about me it’s been the butt of all jokes. I’ve taken it for 15 years.”
A source told the Daily Mail the Mysterious Girl singer has no memory of meeting up with Ms Vardy at a Buckinghamshire hotel.
“Pete doesn’t remember meeting her, never mind anything else,” they said. “He was ribbed by his friends, but they knew the chipolata slur was the opposite to the truth.
“Now he is a dad and has a wife and he just doesn’t think it’s true. He is considering legal action against Rebekah.”
The insider told the paper he is yet to decide whether he will go through with it, “but he is very upset”.
Addressing the issue on Loose Women recently, Mr Andre told the panel he is fed up with it being brought up again.
He told the panel: “It’s not the fact that something was said, because we all say stupid things. It’s because it’s brought up again and again and again, at some point you kind of go “OK”.
“It becomes more than just that. OK, we all say stupid things, everyone does. I’m sure I’ve said many stupid things in my life.”
Ms Vardy told the High Court during her libel trial she apologised to Mr Andre for the remark on his manhood.
She said she had sent Mr Andre and his wife “personal messages a couple of years ago” to address the comments made in the News of the World article.
She also alleged it included “a lot of things in there that didn’t come out of my mouth, that were misrepresented.”
She added: “It’s one of my biggest regrets” telling the court she was “very young at the time.”
The Independent approached Mr Andre’s representatives for comment.