Early data suggests more than 10 per cent of Victorian dairy farmers chose not to renew their license for the 2020/21 season, but industry analysts remain confident for the season ahead thanks to good rainfall and retail sales.
Dairy Australia's Situation and Outlook report for December found preliminary data indicated about 11pc of dairy farmers in Victoria had not renewed their license.
The figure is higher than the rate of exits in 2018/19 and is a significant increase compared to the long-term trend of 4pc.
In its report, Dairy Australia noted that while exits could represent both consolidations and the removal of productive capacity, that figure would no doubt put an upper limit on milk production growth for this and future seasons.
Simon Greaves runs a mixed operation at Nerrena in South Gippsland.
Mr Greaves currently milks 250 head but plans to drop back and focus on beef.
He is preparing to take a week off for the first time in 20 years, after farming 850 acres seven days a week.
He said the intensive labour required for dairy made it less attractive compared to other industries.
"They've got to make dairy sexier," he said.
"It's always been looked down upon."
Mr Greaves said attracting people to the industry required better incentive for workers, such as better rosters and pay, but that was not possible if farmers themselves were not making reasonable money.
He said young people looking to enter the industry were few and far between, with most lacking the upfront capital needed to buy adequate land.
His son Josh is a sixth-generation farmer and he says he loves the lifestyle but there needs to be more support for young people looking to enter the industry and more agriculture education in schools.
"I can guarantee most kids at my high school came from dairy farms or their parents worked for the dairy industry," he said.
"Everyone around here is supported by our industry."
He knew of only one other person his age who was looking to get involved in dairy farming.
"No one has the work ethic these days," he said.
"Who wants to get up at 5am [and work until] 7pm, seven days a week?
"Christmas, Boxing Day, every holiday - we don't get paid extra for our milk if it's a public holiday."
He hoped the continued exit of farmers from the industry would have a positive effect on prices.
"If so many people are getting out they're going to eventually have to pay us more for our milk," he said.
Bass dairy farmer John Morgan retired from the industry about four years ago, when the average age of dairy farmers was around 65-years-old.
There were 23 dairy farms in the area when he started farming 47 years ago.
Now, only three remain, with the majority of farms switching to beef.
He said while he enjoyed it at the time, uncertainty around markets could make dairy farming difficult.
"When I started doing it it was a lifestyle, when I left it was a business," he said.
"For a number of years we got told to increase production, which we did.
"The factories were virtually getting us to produce cheap milk but they didn't have the markets for it, so they just paid us less.
"We ended up with a very good herd and the last few years we made more money around stock sales than we did out of milk."
The Dairy Australia report also found securing farm labour was an ongoing issue for the industry.
The pool of international workers, including both skilled and casual 'working holiday' staff, had dried up as a result of closed national borders, while internal border closures added further challenges.
Australian Dairy Farmers chief executive officer David Inall said securing access to a reliable, long-term workforce remained one of the biggest hurdles.
"Unfortunately, we don't see that changing any time soon," Mr Inall said.
"But ADF has had some policy wins over the past couple of years in this area.
"We lobbied the federal government to change the Dairy Industry Labour Agreement to allow skilled migrants working on dairy farms to apply for permanent residency."
Despite the challenges, Dairy Australia noted production had increased in western Victoria and Gippsland.
It comes as national milk production is expected to lift in the 2020/21 season, for the first time in three years, although it is tracking towards the bottom end of the forecast range of 1-3pc growth.
Production growth has been greatest in South Australia, western Victoria and Tasmania but is still in the negative range for most other regions.
Even so, industry analysts are confident this season will be one of the best for the dairy sector in many years, thanks to good rainfall and improved retail sales.
The report found the easing of COVID-19 restrictions "unleashed months of pent-up consumer demand" and saw food service sales rebound since spring.
Sales of milk and butter grew 4.6 and 8.7pc respectively in the year-to-November, although consumers have shifted to buying larger-pack sizes that are usually priced at a discount.
Mr Inall said the hope was the lift in domestic sales would continue.
"Australian dairy is fresh, it is nutritious and it is a product that we are all very proud of," he said.
He said the Australian Dairy Plan set the goal of boosting milk production to 9.6 billion litres by 2025 and to 10 billion litres by 2030.
He said if the plan was successful, it would result in re-investment on farm.
The report also noted milk production in the four-largest dairy exporting regions increased for more than four months in a row, the first time since 2014.
Market analysts have attributed that to the tight affiliation Chinese consumers have with Australian dairy, particularly infant formula, but the current relationship tensions between the two countries was making dairy exporters nervous.