Position an oven rack in the middle of the oven and heat the oven to 350°F. Generously butter and flour a 10x2-inch fluted quiche pan or a 9x2-inch round cake pan.
In a medium bowl, whisk together the flour, baking powder, and salt; set aside. In a large bowl, beat the butter with an electric mixer until smooth. Add 1 cup of the sugar. Split the vanilla bean (if using) with a paring knife and scrape out the seeds. Add them to the bowl; reserve the pod. Beat on medium speed, scraping down the sides, until the mixture is well combined and has some body, about 3 minutes. If using vanilla extract, beat in 1 tsp. now. Add the eggs, one at a time, beating well after each addition. Add half of the flour mixture and mix on low speed until just blended. Add the milk and mix until just blended. Add the remaining flour mixture and mix until just blended.
Pour the batter into the prepared pan and spread evenly. Arrange the peach slices, overlapping slightly, on top of the batter in a circular pattern (you might not need all the slices) and sprinkle with the remaining 2 tsp. sugar. Bake until a toothpick inserted into the center of the cake comes out clean, 50 to 55 minutes.
While the cake bakes, make the glaze. Put the apricot jelly, brandy, and the empty vanilla bean pod (if using) in a small saucepan and heat on medium low until syrupy. Remove from the heat, cover, and let the vanilla bean infuse for 20 minutes. Remove the bean. If using vanilla extract, stir in 1/4 tsp. now.
When the cake is done, let it cool on a wire rack for 20 minutes. Run a small knife around the inside edge of the pan to loosen the cake. Remove the cake from the pan; transfer it to a serving plate. Just before serving, reheat the glaze to liquefy it and brush it over the peaches.