ZAMBOANGA CITY, Philippines (Reuters) - Fighting between security forces and rogue Muslim rebels seeking to declare an independent state escalated in a southern Philippine city on Thursday and spread to a second island, officials said.

U.S.-trained commandos exchanged gunfire with a breakaway faction of the Moro National Liberation Front (MNLF) holding dozens of hostages in Zamboanga City, on the southernmost island of Mindanao, army spokesman Domingo Tutaan said.

The violence illustrates the security challenge potential investors face in the impoverished south of the majority Roman Catholic country despite a strong nationwide economic performance in the second quarter.

It also raises questions about the strength of a peace deal agreed last October with a larger Muslim rebel group, the Moro Islamic Liberation Front, to end four decades of conflict that killed 120,000 people and displaced two million in the resource-rich region.

The Muslim Moro make up the largest non-Christian group in the Philippines, at around 10 percent of a total 97 million Filipinos.

Many houses were on fire on Thursday as soldiers retook a district that rebels occupied for three days, a Reuters witness said. Loud explosions and sporadic gunfire were heard as rebels shelled government positions.

Troops also made house-to-house searches after expelling rebels for possible booby-traps, moving cautiously to avoid sniper fire. Officials said security forces suffered no casualties. They did not know about rebel casualties.

The violence spread to the smaller island of Basilan where soldiers fought for nearly two hours before turning back 150 gunmen even as troops shelled MNLF bases, Tutaan said.

"We don't want any civilian casualties," Tutaan told a news conference in Manila. "... (we) want that this incident in Zamboanga to be resolved immediately or as soon as possible."

Three soldiers were killed and nine wounded in the Basilan battle, he added.

Zamboanga Mayor Isabelle Climaco-Salazar said she had opened a communication line with MNLF founder and breakaway faction leader Nur Misuari, appealing to him to free hostages, particularly women and children.

The standoff has paralysed Zamboanga with as many as 170 civilians believed to be trapped, if not physically held hostage.

Schools and offices were closed for the fourth day, but some shops and banks reopened. Flights and ferry services were also suspended. Nearly 13,000 people have been displaced in five districts of the port, known as the city of flowers.

(Additional Reporting By Manuel Mogato in MANILA; Editing by Nick Macfie)