The names of the four men have not been released though they were seen being led to a court on the Greek island of Lesbos.
Authorities said that the suspects were later returned to police detention and given three days to prepare their testimony.
The fires swept through the overcrowded camp at Moria on two nights last week, prompting more than 12,000 migrants and refugees to flee. Nobody was hurt.
Most of them remain without shelter, camped on a roadside near the gutted camp.
Two other migrants allegedly involved in setting the fires are minors who are being held by police on the Greek mainland but have not been formally charged.
They were among around 400 unaccompanied minors from Moria who were flown to the mainland after the fires.
The Greek government maintains that the fires at Moria were set deliberately by migrants protesting confinement after the site was locked down due to an outbreak of Covid-19.
Athens on Tuesday urged the European Union to jointly run new refugee camps on Greece’s eastern islands as part of an overhaul of the EU’s migration policies.
“The images of the Moria camp are a painful reminder of the need for Europe to come together,” European Commission President Ursula von der Leyen said Wednesday during her state of the union address.
“Everybody has to step up here and take responsibility — and the commission will do just that,” von der Leyen told lawmakers at the European Parliament, adding that the commission was working on a plan for a new camp on the island.
“But I want to be clear: if we step up, then I expect all (EU) member states to step up, too. Migration is a European challenge, and all of Europe must do its part," she said.
Greek army personnel have been building a new camp on the island, using tents for emergency housing for asylum-seekers left homeless by the fire.