The history of Ambon began at the fort construction on the island as a place to trade and spread religion by the arrival of Portuguese. They set up the city during the political conflict between the rules of the Sultanate Ternate and Islamic society of the north Hitu beach. In 1512, the first Portuguese came in Ambon with his eight crews was Fransisco Serrao. Their ship wrecked in the Nusa Penyu pass and was stranded at Nusa Telu (Island Three) in front of Asilulu village, a western end of Hitu. They were accepted by the King of Hitu Meseng Village. Rumphius and Valentjin claimed that they stayed near Pikapoli river in the south of Mamala Village. Portuguese was refused by the Muslim population of Hitu Village due to their religious differences and their disrespectful and rude behavior.
The first fort of Portuguese was established in Hilla Kaitetu at north Hitu Beach on May 20th 1569 by the Admiral Goencalo Peirera Maramaque after having conquered the Leihitu Peninsula. Their second fort was set up between Galala village and Hative Kecil village surround Wai Tua sea located in front of Martafons Cape and Rumah Tiga Village called Hukunalu. Moreover, they built the third fort near Halong, Teluk Ambon area since the second fort was burned down.
The fourth Portuguese fort was created by Sancho de Vasconcelos. During the construction of this fort, Nusaniwe, Kilang, Soya, Hutumuri, Puta and Ahusen village which was loyal to Portuguese became unfaithful and against the rules. The construction was obstructed and ended in 1576.
All people including Hatiwe, Tawiri, Halong and Mardjiker people, also Portuguese community named Casados whom married indigenous women moved to the new fort and created their houses surround the Castle or fort Kota Laha since the fort had been completely finished. A new small village beside the fort was settled in 1567. Therefore, July 1576 was indicated as fort and Ambon city officially announcement. The missionary around the public house grew up on a farm to produce the foodstuffs. In addition, the infrastructure of the city was four churches, a hospital, a meeting hall, and a port of north fort. The city border area of Portuguese era was divided into the west to east that was Wai Batu Gajah till Wai Tomu andthe south border area was through the street known as Said Parintah, Kapitan Ulupaha and Benteng Kapaha street. (Leirissa et al, 2004: 19-28).
In 1602 The Admiral Andre Furtado de Mendosa came to Ambon to change Admiral Vasconsalo. He used the fort Kota Laha as a Portuguese fortress and central government in Ambon and surrounding. The next head of the government was 1) Gonsallo Pereiro, 2) Johan Caijadoe, 3) Steven Texeira, 4) Gaspar de Mello.
On February 25th 1605 , the castle or Kota Laha fort was handed over to the Dutch Admiral Steven van der Haghen under the government of Gaspar de Mello through an agreement between two leaders without any battle.
During the rule of Steven van der Haghen, the Dutch people successfully took over the castle from Gaspar de Mello and renamed it into Victoria Castle or glory fort.
