Set aside a few of the blueberries for garnish, if desired. Lightly grease a small (4-cup) bowl with cooking spray. Line with 2 long pieces of plastic wrap, letting the excess hang over the sides. Make room in the fridge.
Finely grate 2 tsp. zest from the orange. Squeeze 2/3 cup juice from the orange. Put the juice and zest in a medium saucepan with about two-thirds of the blueberries and the sugar. Bring to a boil over medium heat, stirring. Reduce the heat to low and simmer, stirring, until the berries are soft and the liquid is syrupy, about 3 minutes. Slide the pan from the heat and add the remaining blueberries. With the back of a spoon, press on the fresh blueberries until lightly crushed. Set aside to cool slightly.
Arrange one row of ladyfingers top side down and slightly overlapping (keep attached, if possible) to cover the interior of the bowl completely. Fill gaps with smaller pieces. Spoon the berries (with as little liquid as possible) into the bowl. Cover with the remaining ladyfingers, then pour the juice on top. Wrap the excess plastic over the bowl. Put a flat plate large enough to cover the pudding on top and set a 2- or 3-pound weight (think large can of tomatoes) on it. Refrigerate for at least 8 hours or up to 2 days.
To serve, unwrap the mold and invert onto a shallow serving bowl or plate. Gently pull on the plastic while lifting off the bowl. Cut into wedges. Garnish with the reserved blueberries, if you like.