The fort was established by King Rudra Deo in 1054. The city is known for its cultural heritage as it is the place where Lord Rama killed the demon Taraka and also the site of sage Vishvamaitra's Ashram. Buxar district has close linkage with that of its parent district Bhojpur and has an old history. According to mythology ancient significance of Buxar is mentioned in epics like Brahamana Purana and Varah Purana. In 1764 the battle of buxar was fought near this fort which resulted in the victory of East India Company.

During the excavations held during 1926–1927 along the river bank they unearthed two seals with inscriptions in the early Brahmi script that date back to the 3rd century and 4th century, indicating that the mound is quite ancient. In 1812, Francis Buchanan visited the fort. He stated that only the southern side of the fort and bastions standing. He also mentioned that the fort had a subterranean passage which housed ancient images and that place is known as Patalganga. In 1871–1872 Alexander Cunningham visited the fort, he stated that he did not find any historical artifacts there and also said that it was “a purely Brahmanical site” and that it had “nothing of archaeological interest.”