During a visit to the electorate to campaign with Rebekha Sharkie, Mr Katter said Ms Downer had emerged from "extreme right-wing politics, dangerously right-wing sort of areas".
He said he might be characterised in a similar way but for his focus on helping the poor.
Ms Downer's father, Alexander Downer, held Mayo for the Liberal Party from 1984 until 2008.
Mr Katter is also the child of a politician but insisted he had not been helped into parliament.
"I've made my own way in life," he said.
Ms Sharkie won Mayo from the Liberals in 2016 but was forced to resign earlier this month after being caught up in the citizenship saga.
The two cross-benchers toured a men's shed in the Fleurieu Peninsula town of Yankalilla on Wednesday before holding a question and answer session at a nearby community centre.
Mr Katter praised Ms Sharkie's grassroots style and questioned whether Ms Downer had earned the backing of the local community.
"If I'm looking to put someone up for parliament, I want to know what they've done for their community," he said.
Ms Sharkie again apologised to Mayo voters for sending them back to the polls.
"I, hand on heart, put in my documents in April 2016 - long before they even thought of having a double dissolution, and I didn't spend a single day in the parliament as a dual citizen," she said.
Mr Katter and Ms Sharkie disagreed at times during the hour-long session on Wednesday, including on the issue of how many visas should be issued.
But she said that was the strength of the crossbench.
"We all come together with different views on a whole range of issues where we both proudly support Australian-made, but how you get there is a little different," she said.
Ashley Edwards, 35, who attended the forum, said support for Ms Sharkie in the region was strong.
"I'd be very surprised if she didn't get back in," he said.
"She's kind of like your next door neighbour but she speaks up and that's what this area really needs.
"I thought it was an odd pair-up but the independents are helping each other out and that's really good to see."
Australian Associated Press