Prince William and Kate Middleton may have found their “forever home,” but it’s reportedly coming at a cost to their neighbors. A palace spokesperson spoke to People on Aug. 16 and confirmed that “the Wales family will move house later this year,

Kate Middleton, Prince William and their children are moving to Forest Lodge later this year, and two families living in cottages near the residence were asked to vacate and have since relocated ahead of the royals' move.

But soon after the family started to unpack, things changed again. Queen Elizabeth died, just as Kate Middleton and Prince William landed in the dwelling. “It was not exactly the settling-in period they had hoped for,” a source said of that period in the Waleses’ lives.