Russian/Baltic architecture along the grid of old town's streets, realized during the Russian occupation of the city in 1878–1918, singles out the city in Turkey. Fethiye Mosque (Fethiye Camii) in the city centre, originally built by Russians in the occupation period as a church, is the only mosque in Turkey having that distinctive architectural style.
The Church of Apostles just below the castle, now known as Kümbet or Kethuda Mosque (Kümbet/Kethuda Camii), is also well worth a look. The building was originally an Armenian church built in 10th century, and upon capturing the city, Ottomans converted it to a mosque in 1579. Later, when Russians came over, it became a church again, this time serving Russian Orthodox believers. After the Turks took back the city, it served for non-religious purposes for a time (such as a warehouse), and in 1998 consecrated as a mosque again.
Friday Bath (Cuma Hammam) by the river - you can see it from the castle. Completely derelict but worth a look inside if you don't mind pigeons and a lot of broken glass.