The Victorian Farmers Federation has called for the axing of stamp duty, saying it's preventing young people from buying into the market.
VFF president Emma Germano said stamp - or land transfer - duty should be replaced by raising the GST.
Ms Germano told a state parliamentary committee that back in 1999 plans were put forward to replace a number of inefficient state-based taxes with the GST.
"That job was unfortunately never finished, and we are still stuck with taxes like stamp duty and payroll tax that impact business, drag on international competitiveness and act as a barrier for young farmers to buy farmland," she said.
A state parliamentary committee is looking at issues around land transfer duty fees and its instruments within the Duties Act 2000.
The inquiry will analyse the current situation regarding the land transfer duty tax, examine interstate and international models and explore potential alternatives to land transfer duty.
In Victoria, stamp duty is based either on the purchase price or the property's open market value (whichever is greater).
The rate is determined on a sliding basis, starting at 1.4 per cent for properties valued at less than $25,000 and increasing to 5.5 pc for properties valued at, or more, than $960,000.
Ms Germano said an expanded GST was the best option to replace stamp-duty
Farmers rejected - as a long-standing principal - that a broad-based land tax should be imposed on productive farmland.
"Land is the working capital of every farm business and we know that land taxes have a disproportionate impact on the agricultural industry which in turn affects food affordability and reliability for the entire community," Ms Germano said.
READ MORE: Farm prices continue to stun
She said Victoria should focus its attention towards working with the commonwealth and other states to finally clean up the tax system.
Ms Germano said the VFF also called for an increase to stamp duty exemption for young farmers to help them get a start and to keep up with movements in the rural land market.
"The current exemption threshold of $600,000 land value is far too low with the median farmland price in Victoria at over $10,000 per hectare," she said.
"That doesn't buy much for a young farmer looking to go out and start their own business.
"The VFF believes the exemption threshold should be raised to $1 million, with a concession on the next $500,000 thereafter."
Victorian Young Farmers secretary Linda Thompson runs a small beef operation, near Hamilton.
She said axing stamp duty would be "very helpful.
"It [stamp duty] probably tips it over the edge," Ms Thompson said.
"Where there might be some affordability then you start adding that to it, it tips it over."
She said taking on a family farm also involved many rules and regulations, as well as paying out siblings.
The other impact on affordability was developers buying up land on the outskirts of cities and towns.
Taking over properties from parents, or other family members, was also resulting in a shrinking of the pool of land available to buy.
"There are a lot of family farms around here which are well established and debt free and the owners are purchasing other farms - we are going to have more acreage per farmer, but a lot less people living here."
She said one small farm near Hamilton recently sold for $700,000.
"But you can't make a living off it, because it's only a small type hobby farm," she said.
"The land price around here has risen.
"It used to be up to $3000 an acre (0.4 hectares), now it's up to $7000-8000 an acre."
"Even though cattle prices have gone up, what you need is land," she said.
"A lot of people are looking at careers off farm, they might have jobs elsewhere."
Nikki and Paul Hurn have a small livestock operation at Kingston.
They are keen to expand but are finding stamp duty and the price of land have "skyrocketed" since the COVID-19 pandemic.
"It's pretty much made it near impossible for young people to actually afford land," Ms Hurn said.
"Land you might have bought for $2000 an acre - and that was expensive - a few years ago is now going for $8000-9000 an acre.
"Unless you are inheriting a family farm, there is no way of bypassing that, so the stamp duty on top of that is an extra cost you have to factor in."
She said she and Paul owned a house in Creswick but there short term goal was to buy some agricultural land.
They wanted to buy a small property, "nothing too big", to run a few sheep and cows.
"Something that would have been realistic for us, maybe five years ago, has quadrupled in price in this area," she said.
Land was also tightly held by the current owners.