Buddhist Caves are rock cut caves used as monastic quarters earlier. These caves are over 2000 years old and were carved during the reign of Emperor Ashoka. The oldest is the Khapra Kodiya caves. The walls of this cave have various scribbles and cursive letters dating back to 3rd – 4th century. It is the unadorned cave amongst all and is considered the earliest monastic settlement in the area.

The group of caves near to the Modhimath are known as Baba Pyare cave. The north group of it has four caves. The next set of caves lies little to the south of the eastern end. The caves were constructed during the Satyavahana regime and are influenced by Buddhist architecture. Baba Pyare caves has oeuvres of both Buddhism and Jainism. Uperkot caves are situated within Uperkot fort in the north western part of Jama Masjid close to Adi Kadi vav( one of the two step well). They were carved in 2nd–3rd century A.D is influenced by Satvahana architecture with the combination of Graeco- Scythian style.