Located at the busy junction of Chulia Street and Jalan Masjid Kapitan Keling, this mosque's spacious grounds and gardens is a serene spot for many to watch the world go by. A monumental mosque crowned by yellow domes and minarets, it is the largest historic mosque in town founded in 1801 by an Indian Muslim merchant, Cauder Mydin Merican. It was originally built as a rectangular building with verandahs and a hipped roof and had been enlarged several times. British architects commissioned by the Muslim and Hindu Endowments Board gave the mosque its British Moghul Revival appearance in 1916.

The name of mosque was taken from the Kapitan Kelings, people who were appointed leaders of the South Indian community by the British. The term 'keling' derived from the ancient Hindu kingdom on the Coromandel coast of South India. It was generally used to denote all those who came from there. As the Indians found it difficult to pronounce certain English words, the title "Captain" was somehow transformed into "Kapitan". From there, the Kapitan Kelings (or Captains of the Kelings) came about.