In a mixing bowl, combine the chilled Earl Grey tea, milk, heavy cream, sugar, vanilla extract, and lavender extract (if using). Stir well until the sugar has dissolved completely.
Taste the mixture and adjust the sweetness according to your preference. Add more sugar if desired.
Pour the mixture into the popsicle molds, leaving a little space at the top for expansion. If your popsicle molds come with a lid or cover, secure it in place. Otherwise, insert popsicle sticks into the molds.
Place the popsicle molds in the freezer and let them set for about 1 hour. After an hour, remove the molds from the freezer and gently insert the popsicle sticks further into the partially frozen popsicles. This will help keep the sticks in place.
Return the popsicles to the freezer and let them freeze completely for about 4-6 hours or overnight.
Once the popsicles are fully frozen, remove them from the molds by running warm water over the molds for a few seconds. This will help loosen the popsicles.
Enjoy your homemade London Fog popsicles immediately or store them in an airtight container or freezer bag in the freezer for later consumption.