The Drostdy was built by the Dutch East India Company in 1747 to serve as residence and official headquarters for the Landdrost. Soon after a gaol, a house for the secretary, a mill and various outbuildings were erected.
The first Landdrost to be appointed to this district was Johannes Theophilus Rhenius and he was assisted by a board of burger heemraden and subordinates like secretary and a gaoler as well as many slaves. From 1827 the Drostdy was occupied by the civil commissioner who, with the resident magistrate, replaced the board of Landdrost and heemraden when they were abolished by the British colonial government.
In 1846 the government sold the Drostdy and the property was subdivided. In 1855 the former Drostdy was bought by the Steyn family and it remained in the hands of this family until 1939 when it was bought by the government of the Union of South Africa for the purpose of establishing a museum.