The head of Lebanon's Red Cross, George Kettani, said at least 100 people had been killed.
"We are still sweeping the area. There could still be victims. I hope not," he said.
Kettani earlier told broadcaster LBCI that the Red Cross was coordinating with the health ministry to set up morgues because hospitals were overwhelmed.
Hours after the blast, which went off shortly after 6pm (1500 GMT), a fire blazed in the port district, casting an orange glow across the night sky as helicopters hovered and ambulance sirens sounded across the capital.
The blast was heard throughout Cyprus, which is about 100 miles (160 km) away.
It revived memories of a 1975-90 civil war and its aftermath when Lebanese endured heavy shelling, car bombings and Israeli air raids. Some residents thought an earthquake had struck.
Dazed, weeping, and injured people walked through streets searching for relatives.
"The blast blew me meters away. I was in a daze and was all covered in blood. It brought back the vision of another explosion I witnessed against the US embassy in 1983," said Huda Baroudi, a Beirut designer.
Prime Minister Hassan Diab promised there would be accountability for the blast at the "dangerous warehouse", adding "those responsible will pay the price".
The US embassy in Beirut warned residents about reports of toxic gases released by the blast, urging people to stay indoors and wear masks.