Witnesses told local media of hearing at least six shots coming from outside, and seeing Cali's family members rushing outside and crying next to his body.
Now, there is concern that this could be the start of a mob war in New York City.
The 53-year-old reportedly ascended to the top spot in the gang in 2015, taking over from Domenico Cefalu.
"Within recent months one of John Gotti brothers, Eugene Gotti, came out of federal prison".
Cali kept a much lower profile than Gotti and was killed in far less spectacular fashion than Castellano.
According to reports, Cali's death marks the highest ranking member of a NY crime family to be murdered in decades.
Alleged Gambino Crime Family Boss Frank Cali Shot Dead Outside Home
Cali becomes the first Mafia boss murdered in NY since Paul Castellano was gunned down outside Sparks steakhouse in Midtown in 1985. Gotti then took control of the family. He was pronounced dead after being taken to Staten Island University Hospital. "But I guess old habits die hard".
The Gambino family was once considered the biggest organised crime group in the USA, the New York Times reports, but began to decline after Gotti and other senior figures were jailed.
After prosecutors called him "a danger to the community" Pandrella, 59, pleaded not guilty in Brooklyn federal court to accusations of shooting Zito in his own home and stealing money from him.
Last October, 71-year-old Sylvester Zottola of the Bonanno family was shot dead while waiting in his auto for an order from a Bronx McDonald's.
Cali had been considered a unifying figure in the years after then-Gambino boss John Gotti, "Dapper Don", was convicted of murder and racketeering in 1992 and sent to prison for life, CNN affiliate WPIX reported.
In that case, authorities intercepted conversations shedding light on his quiet underworld command. During a bail hearing in February 2008, Assistant US Attorney Joey Lipton noted that "Cali is seen as a man of influence and power by organized crime members in Italy".