Position a rack in the center of the oven and heat the oven to 400°F. Line a baking sheet with kitchen parchment. In a small saucepan, combine the water, butter, sugar and salt. Bring to a boil over medium-high heat. Add the flour and stir vigorously with a wooden spoon until smooth.
Remove from the heat and beat with a wooden spoon just until the steam stops rising, about 1 minute. The mixture will smooth out and pull away from the sides of the pan.
Add one egg and beat well with the wooden spoon. The batter will seem to break apart, but keep working until it becomes smooth.
Add the second egg and beat again until the mixture is completely smooth.
Scoop the batter into a pastry bag fitted with a 1/2-inch plain pastry tube (Ateco #5). Hold the bag about 1 inch above a parchment-lined baking sheets and pipe out mounds about 1 inch in diameter. Beat the egg yolk with the milk to make an egg wash. Lightly brush the egg wash on top of each puff with a pastry brush, tapping down any points of dough.
Bake until the pastries are puffed and deep golden, 22 to 24 min. Let cool on a wire rack. If not using the same day, transfer to a plastic bag and freeze for up to four months; thaw in the refrigerator overnight before using.