Friends said 35-year-old Simon Nellist was an experienced diving instructor who “loved the water”.
It is believed Mr Nellist was a British expatriate living in the Wolli Creek area of Sydney, and had been due to get married.
Emergency services were called to Little Bay at about 4.35pm local time on Wednesday after reports a swimmer had been attacked by a shark.
Human remains were later found in the water, according to reports.
Mr Nellist is said to have been due to compete in a charity swim this weekend, but the event has been cancelled out of respect.
The diving instructor’s friend, Della Ross, was among those to pay tribute after the attack.
She told Australian broadcaster 7News: “Everything that is connected to Simon, to me is connected to the ocean.
“The news hit us like a truck because he is really one of the people that makes this earth better.”
Witness Kris Linto said the swimmer was in the water when the shark “came and attacked him vertically”.
He told Nine News TV: “We heard a yell and then turned around, it looked like a car just landed in the water, big splash.”
Lucky Phrachnanh, from New South Wales state ambulance, said the victim “suffered catastrophic injuries as a result of the attack”, adding there was nothing paramedics could do when they arrived at the scene.
The incident is believed to be the first fatal shark attack in Sydney since 1963.
Several Sydney beaches, including the iconic Bondi and Bronte, were shut down in the aftermath of the incident on Thursday.
Drum lines, which are used to bait sharks, have been set up near the attack site while drones have been deployed as officials search to see if the shark is still in the area.
Dylan Parker, the mayor of Randwick Council, which includes Little Bay, said: “This has been a complete shock for our community.
“Our coastline is our backyard and to have a tragic death under such horrifying circumstances is completely shocking.”
Organisers of the Murray Rose Malabar Magic Ocean Swim had considered postponing Sunday’s swim until March, but said they decided to cancel it after consulting with the local council and lifesavers.
A spokesperson for the New South Wales Department of Primary Industries said its shark biologists believed a great white shark at least three metres (9.8 feet) in length was likely responsible for the attack.