Welcome to Fife and Drum Inn A Historic Williamsburg VA Boutique Hotel The most conveniently located, bed and breakfast in downtown Williamsburg VA. Fife and Drum Inn is located between the Colonial Williamsburg Historic Area and the College of William and Mary. From the Inn’s front door, the College Admissions Office is just a two-block walk west, and the colonial Governor’s Palace is a two-block walk east. Our B & B is located on the north side of Merchants’ Square, a two-block area that contains some of the best restaurants in Virginia and more than thirty specialty retail shops.