Preheat oven to 350º F and lightly grease a 9×13-inch baking dish with butter or non-stick spray. Once roasted potatoes are cool enough to handle, remove skins and place potatoes in a large bowl.
Pour in milk, brown sugar and butter, and mix in cinnamon, nutmeg, ginger (if using) and allspice. Mash together until potatoes are smooth and everything is evenly incorporated.
Transfer mixture to baking dish and set aside.
Melt butter in a medium saucepan over medium heat and stir in brown sugar, whisking until sugar is dissolved. Stir in (crushed) pretzels, then pour mixture over sweet potatoes. Top with an even drizzle of caramel sauce, then sprinkle with sea salt.
Place baking dish in oven and bake for 25-30 minutes, or until heated through.