Bill and Marian Egan and their three kids were living in a small house at Leongatha when Mr Egan devised a plan in 1982 to establish his own stock and station agency with a fellow agent, Michael Stevens.
At the time, Mr Egan was working for DSM Estates, previously known as Dennys Strachan Mercantile and Dennys Lascelles, as the lead stock agent and auctioneer at Korumburra.
"We needed money to start," Mr Egan said.
Born and raised in Melbourne, Mr Egan said people used to watch him in the yards as a young agent and ask "if I knew what I was doing".
"When I started work in 1968, I had no perception about what I was going into," Mr Egan, who attended St Bede's College at Mentone, said.
"I tried to get a job with Australian Estates and Dalgety's in their trainee programs, but I couldn't get in.
"I managed to get a job with Farmers & Graziers Co-Operative, and I got my start, a start that everybody needs."
It was not until after his 18th birthday that Mr Egan had stepped foot into a saleyard.
From the get-go, he was instantly challenged compared to other colleagues who had grown up in the bush.
"When I started work in 1968, I was on $25 a week," he said.
His early stock agent days were spent at the Newmarket and Dandenong yards, and before long, his leadership potential was recognised.
"It was about that time that Dan Mehegan came to Farmers & Graziers from Dalgety's, he was a terrific bloke, a fabulous auctioneer and he just embraced everybody," he said.
"The old-school attitude at Newmarket was that the senior blokes would arrive, look at the livestock and sell them, then go to the pub and go home.
"When Dan started, he said we would arrive together, we would work together and we would leave together."
After four years in Melbourne, Farmers & Graziers offered him a position in the Mallee town of Berriwillock and a year later in 1973, Mr Egan married the "office girl" who he had met in Melbourne.
"Marian was a Melbourne person and she worked as an office girl for Farmers & Graziers, and luckily she got a job at the Sea Lake High School so we were both employed," he said.
Only a year later, Mr Egan was offered a job as head stockman at the company's Leongatha branch.
He has remained in South Gippsland since.
"I'd sell a few old cows and a couple of vealers at the fat market on a Friday in Korumburra," he said.
As the town became a prominent selling hub in the 1970s, Mr Egan's business and reputation grew, and in 1978, he joined the iconic agricultural western district firm of Dennys Lascelles.
It was there he developed a close working friendship with Michael Stevens.
In the early 1980s, the pair felt as if opportunities to progress their careers were few and far between, and the concept of establishing an agency was a daunting and difficult process.
"We wanted to start on our own, and that hadn't been done since the days of Scott & Co and we couldn't find any backing," he said.
"I had a little fibre cement house and three little kids, Michael wasn't much better and the bank wanted assets."
Fellow stock agent Terry Johnston, who was managing the local Dalgety branch, "got wind" of the pair's desire to form a new agency and asked Mr Egan about the plans.
"We denied it initially, but Terry said 'If you go on your own, I want to go with you', so we eventually came clean with him," Mr Egan said.
Together the three men went on to establish SEJ, short for Stevens, Egan and Johnston, with the financial backing and support of another stock agent, Bill Sweeney.
From little things big things grow
A local solicitor who owned a farm at Outtrim helped the business secure its first bank loan.
"Peter Boothby was his name and he was an old-fashioned operator who had financed farmers forever in the district," Mr Egan said.
"He used to meet everybody on a Sunday at the farm, so we went down there and had our spiel worked out.
"We gave our spiel to Mr Boothby, and mind you we had a lot of business between the three of us, but in the end he said 'No, I don't think it would work' and he used to say 'run away' which he told us to do."
A few days later, Mr Boothby phoned the three men and after some consideration, organised a meeting with the area manager of the National Bank of Australia.
"Up until this point, we had been trying to deal with Westpac on our own," Mr Egan said.
"He said 'I've rung that bloke at Westpac, and he's no good at all so I've sacked him'."
The NAB went on to approve an overdraft of $100,000 with a private investment of about the same figure.
The firm wasted no time, and in their first sale on November 29, 1983, sold 742 lambs at the Korumburra saleyards, most made between $14-21 a head, to average $20 across the market.
A day later, SEJ sold 405 cattle at Korumburra, to average 72c/kg or $324 a head.
In 2023, SEJ celebrated 40 years and now employs about 20 people.
"From fellas who were just livestock agents with no formal business training, we ran it really well and had a good accountant," he said.
"The four people who started the company, Michael Stevens, Terry Johnston, Bill Sweeney and myself, have never had a cross word.
"And of course you are only as good as the people you have around you, they make you as good if not better than what you really are because good people are the success of all good businesses."
Mr Egan said another highlight was the partnership forged between SEJ and agency LKD (Laffertey, Kyle, Draper) in 2011.
"It was extremely good for both parties, because at the time SEJ were looking to expand and LKD were also looking to make a move away (from Elders)," Mr Egan said.
Mobile phones have been the single-biggest change to the industry in 50 years, according to Mr Egan, who said the life-changing devices meant you could spend more time with your family at night, and less time on the landline when you were home.
Mr Egan, now 73, has formally retired from SEJ and has given the reins onto the next generation of the company.
"We'll introduce the younger members in the company to take over the directorships," he said.
"I loved going to work each day but there comes a time when you must step back."
Mr Egan is now wanting to "wind down" and spend more time with his family.