The gaffe, which drew intense criticism, comes as Tony Abbott battles fury from some voters over his latest budget.
Many see proposals to cut health and education spending as Tony Abbott backtracking on election promises.
Hundreds have staged protests in Australian cities against the budget.
Tony Abbott had appeared on an Australian Broadcasting Corporation (ABC) radio show on Wednesday to defend his proposals.
A woman who identified herself as “Gloria” called in, saying she was a chronically ill 67-year-old grandmother and that she would have to pay more for medical treatment under the new budget.
Tony Abbott winks and grins when the woman says: “I work on an adult s** line to make ends meet, now that’s the only way I can do it.”
He then appears to control himself by pursing his lips, glances at the video camera filming him in the studio, then puts on a serious expression.
Clips of the incident were later circulated online and on national media, prompting criticism that Tony Abbott has lack of empathy for voters.
He told Australia’s Channel Nine on Thursday morning that it was a mistake.
“I shouldn’t have done it… I should’ve been more focused on the caller and less focused on the interviewer,” Tony Abbott said.
He was attempting to engage with the caller and was “momentarily distracted” by the radio show whom he said was “smiling at me and I winked back at him”.
“Mistakes are always regrettable… and I will do my best having made a mistake yesterday to make none today,” Tony Abbott said.
On Wednesday he told another radio station that he was reacting to “an interesting call from someone who had an interesting story”.
Meanwhile “Gloria” contacted the radio show again on Thursday and said Tony Abbott’s actions were “sleazy, slimy”.
It is not the first time Tony Abbott has been accused of insensitivity towards women.
Tony Abbott was accused of being a misogynist by former PM Julia Gillard in 2012 – a label he has strongly rejected.
-1-RnnYLa-oImage source: Wikimedia Commons President-elect Donald Trump celebrated his election victory at the Ultimate Fighting…
Millions of voters across the US chose to return Donald Trump to the White House…
Donald Trump declares victory in the US election as he addresses jubilant supporters in Florida.…
Stocks around the world are rising as Donald Trump appears to be on the cusp…
Donald Trump has won Pennsylvania, North Carolina and Georgia and taken a lead over Kamala…
Quincy Jones, the celebrated musician and producer who worked with Michael Jackson, Frank Sinatra, Ray…