The castle has been mentioned since the 11th century. In 1096, Jeoffroy de Penne accompanied Raimond 5th to the Holy Land. The troubadours and Jourdain, viscount de Saint-Antonin, sang about Adélaïs de Penne.
Of the redoubtable castle which was subject to so many assaults and futile sieges, which sneered at Simon de Monfort, there only remains one romantic silhouette, from the times of the Wars of Religion, rising up from a rocky summit over the river Aveyron. For three centuries, the ruins served as a quarry to build the houses of the village.
