Visingsborg Castle was one of the most impressive 17th century castles and the keystone of the Brahe countship. The foundation stone for the castle was laid in the 1560s by Count Per Brahe the Elder.
Per Brahe the Elder was elevated to the countship in 1561 and the island of Visingsö was conferred on him. Immediately afterwards he began building Visingsborg Castle, and he lived long enough to complete the western wing. His son Magnus Brahe then took over and built the southern wing, which was the castle’s most imposing section, with its halls and staterooms. He also built the eastern wing, which contained the kitchen and household spaces. During the time of Per Brahe the Younger, the northern wing, which was intended for the servants, was built. Material was obtained in part from the Alvastra Abbey, which had been closed down, and in part from newly established brickworks in Gränna. The castle had six towers with gilded spires that rose above the roof and stepped gables. The furnishings included an impressive library and an armoury with weapons for up to 800 men. The ramparts and bastions around the castle were armed with cannons.
A few of the furnishings from Visingsborg Castle are now at Skokloster, but much of the fine rugs, art, weapons and other articles were sold off at auctions in Jönköping at the turn of the 18th century. Silver artefacts were carted off to the Swedish Mint to be melted down.
During the reign of King Charles XII, between 1716 and 1718, the castle housed up to 2,000 prisoners of war, most of them were Russian. Many died during confinement from starvation and privation and were buried in the so-called Russian churchyard a couple of kilometres inland on the island. At Christmas time in 1718, when word of the death of King Charles reached Visingsö, the castle was devastated by a conflagration accidentally caused by the German men on guard.