The mercury dipped to its lowest point this year overnight, reaching as low as -4.5 degrees in Victoria's High Country.
The icy temperatures were due to a high pressure system over the state and a combination of light winds and clear skies.
Inland Victoria recorded temperatures between -2deg and -3deg while Melbourne recorded its coldest start to a day this year and coolest June morning since 2013.
Ballarat dropped to -3.5deg while Coldstream record a chilly -3.6deg just after 6am on Tuesday.
Omeo was the coldest place in Victoria, dropping to -4.5deg while nearby Mount Hotham notched a fresh -3.4deg.
In central Victoria, Bendigo fell to -1.5deg, while Rutherglen, Kyabram and Kanagulk recorded lows of -3.5deg.
Horsham and Wangaratta dipped to -2.9deg while in the Western District, Casterton slumped to -3.1deg and Mortlake -3.4deg while Bairnsdale fell to -0.7deg.
Bureau of Meteorology senior forecaster Chris Arvier said there was a chance parts of Victoria could experience patchy frost on Friday, but not as widespread as earlier in the week.
"We're entering a fairly benign weather pattern for the rest of the week and with dry and cloudy conditions, that will insulate the surface which means the temperature won't fall as low," Mr Arvier said.
"We are expecting a cold front to cross the state this weekend, bringing falls of up to 10 millimetres to mainly the south-west and north-east of the state."
Merino and Hereford producer Kelvin Pendergast, Benambra, said it was the most severe frost the region had received this year.
"We've had worse frosts; a few years ago and the temperature went as low as -8deg but as far as this year goes, it was our worst frost," Mr Pendergast said.
"The country is pretty much in a dormant stage but what green there this, this frost will just brown it off."