The Palace traces its history back to the 14th century, when Jogaila, the Grand Duke of Lithuania, issued an edict donating land in the city to the Vilnius Diocese, for this reason the palace is sometimes referred to as the Bishops' Palace . Construction of the Palace took place in the late 14th century under the auspices of the first Bishop of Vilnius, Andrzej Wasilko, and over succeeding generations, the building was gradually enlarged and renovated. During the Renaissance the Palace was once again renovated, and parks and gardens surrounding the building were expanded.
As the 18th century unfolded, a number of dramatic events in the Palace's history took place: the last Bishop of Vilnius lived in the Palace, Lithuania was annexed by the Russian Empire, and the building itself was badly damaged by two major fires in 1737 and 1748. The Palace was reconstructed in 1750 under the supervision of the architect Laurynas Gucevičius. After its reconstruction the Palace was used as a residence for Emperors, Kings and noblemen. During 1796, Tsar Paul I lived at the Palace. During the course of the 19th century the Palace served as a residence for several Imperial Russian governors, such as Mikhail Muravyov, nicknamed "The Hangman". It was also visited by the future King of France, Louis XVIII in 1804.
In 1812, both the Russian Tsar Alexander I and the French Emperor Napoleon used the Palace as their residence. During Napoleon’s invasion of Russia, he organized military operations and Lithuanian army units from this Palace, including five regiments of infantry, four cavalry regiments, and the National Guard of Vilnius. He received Lithuanian noblemen, newly appointed officials of the administration, and other dignitaries in this Palace as well. After Napoleon's defeat in 1812, the Palace was used for ceremonial proposes; it was here that then-general Mikhail Kutuzov was awarded Russia's highest military award - the Order of St. George. During 1824-1834, the Palace was reconstructed by the prominent St. Petersburg architect Vasily Stasov in the Empire style, under supervision of Karol Podczaszyński. Stasov's reconstruction of the Palace has remained to this day.After Lithuania regained its independence in 1918, the Palace housed the Ministry of Foreign Affairs and the ELTA news agency until it ended up in Poland in 1920. It was restored in the 1930s by Stefan Narębski. After the Second World War, the Palace served as the Military Officers' Centre; later it housed various Lithuanian artists. The Palace was gradually adapted for use as a presidential residence, and since 1997 it has served as the official residence of the President of Lithuania. A flag displaying the coat of arms of the President is hoisted when the President is present in the Palace or in the city.