The Ratu Boko complex sits astride a hill which is part of the northwestern branch of the larger Sewu mountain system, located on the southern part of Central and East Java between Yogyakarta to Tulungagung. In the early 20th century, the Ratu Boko site was believed to be the ruins of a palace (keraton). At that time theu also found a statue of gold-headed man and woman embracing each other. A stone pillar with ornaments of zoomorphic figures among which are elephants, horses, and others were also found among the ruins.

The Abhayagiri Wihāra inscription dated 792 CE is one of the few written evidence discovered in Ratu Boko site. The inscription mentions Tejahpurnapane Panamkarana or Rakai Panangkaran (746-784 M), and also mentions a vihāra located on top of the hill called Abhayagiri Wihāra, which seems to be related to the Abhayagiri Vihāra in Anuradhapura, Sri Lanka.

From this inscription it was inferred that King Panangkaran, who was a pious follower of Mahāyāna Buddhism, in the later period of his reign wished for a spiritual refuge and created a Buddhist hermitage named Abhayagiri Wihāra in 792. A Dhyāni Buddha statue was discovered in the site which confirmed the original Buddhist nature of the site. Nevertheless, some Hindu elements were also found in the site, such as the discovery of statues of Hindu deities; Durga, Ganesha and Yoni.

It seems that the compound was converted as a hilltop fortress by a local landlord named Rakai Walaing Pu Kumbayoni. According to the Shivagriha inscription the place was used as a defensive fortress. The hilltop fortress was used as a fort during a power struggle in the later days of the Mataram Kingdom. The complex consists of gopura (gates), paseban, pools, pendopo, pringgitan, kaputren (womens’ quarters), and a meditation caves.