I did a little research. It turns out that there are two sites in Virginia named Belle Grove Plantation and both are related to president James Madison. I was a bit confused during the podcast which one Len's guest Michelle Darnell was talking about.
The other Belle Grove Plantation Belle Grove Plantation is located near King George, VA is the haunted bed and breakfast that Len's guest was talking about. They have regular tours and what they call haunted night tours.