Cut each banana in half and carefully insert the bottom of each with a wooden popsicle stick, about 1/3 of the way into the banana. Place bananas on a large baking sheet lined with parchment paper. Freeze bananas until partially frozen, about 30 minutes.
Place the vanilla yogurt in a tall pint glass and carefully dip each partially frozen banana into the yogurt to coat leaving 1/2 inch at the bottom of the banana uncoated with yogurt. Line yogurt-coated bananas back on the parchment paper and freeze until completely firm, about 1 hour.
Next, microwave the peanut butter until smooth and creamy, about 30-45 seconds. Drizzle peanut butter evenly over frozen yogurt coated bananas then place on the baking sheet to freeze once last time. Freeze until peanut butter is firm, about 30 minutes.
Enjoy immediately or wrap each banana in plastic wrap and store in freezer for up to 3 months.