After the arrest, Fuller said officers found the body of a 21-year-old woman in an apartment on neighboring Clarence Street.
A 21-year-old woman, now unidentified, was later found dead in a unit block nearby. One woman was stabbed in the back but is expected to survive, he told the newspaper.
Seven Network cameraman Paul Walker - who filmed the 21-year-old man being detained - said the man was "full of blood".
The suspect reportedly shouted "Allahu Akbar" as he stood on top of a auto on Monday afternoon.
Police Commissioner Mick Fuller says the attack "is not now classed as a terrorist incident" and that the man "has no links to terrorism".
Police were investigating a home about 32 km (20 miles) west of the city where the young man lived with his parents.
"They were fearless and I can only use that word seriously - they were significantly courageous people", he said.
"A number of members of the public physically restrained the offender and I want to acknowledge those people", NSW Police Superintendent Gavin Wood said.
The man has been taken to Day Street Police Station.
The injured woman was taken to hospital in a stable condition.
A Sky News reporter and other media reports said the man shouted "shoot me" at police.
His body was pinned down with the two chairs while a milk crate was used on his head and neck until police arrived.
The alleged attacker was subdued by members of the public before being taken into police custody shortly after 2pm.
Ms Megan Hayley said a man with a large kitchen knife chased several people though a busy office district and "five or six others were chasing him behind, trying to stop him, they caught him and restrained him" with chairs and a plastic crate.
"To approach a person with a mindset of obviously what this person [had], with clear evidence of a stabbing previously, these people are heroes".
In the video, the man, with a bloodied shirt, jumps on a auto and shouts something that is hard to hear. She was found in a unit block above Clarence St Cyclery, which was forced to close after police set up a crime scene in front and behind the shop.
"So insane! A random dude just started stabbing people in the city right outside where we were having lunch".
Cuthbert and his friends Paul and Luke O'Shaughnessy, all from Manchester in the United Kingdom, and Alex Roberts, from Cronulla, were shaken after the incident but said they did not hesitate once they saw the incident unfolding.
The driver of an Uber has described the terrifying moment the knife-wielding man jumped on top of his vehicle.
A man has been dramatically arrested in Sydney's CBD after going on a stabbing spree in Sydney's CBD.