You can fast travel to tracked quest location, I think as long as it’s not a new location. On Xbox you open the main menu/wheel thing, hover over the quests option at the bottom and just press x.