Bring milk and tea to a boil in a small pot over med-high heat. Turn heat off and steep for 10mins. Strain and measure out 1 cup. Top up with milk if needed.
Preheat oven to 350F. Grease and flour three 6" cake rounds and line with parchment.
In a medium bowl, whisk flour, baking powder, spices, and salt until well combined. Set aside.
Using a stand mixer fitted with a paddle attachment, cream butter and sugars on med-high until pale and fluffy (approx 3mins).
Reduce speed and add eggs one at a time fully incorporating after each addition. Add vanilla.
Alternate adding flour mixture and Chai milk, beginning and ending with flour (3 additions of flour and 2 of milk). Fully incorporating after each addition.
Bake for 35-40mins or until a toothpick inserted into the center comes out mostly clean.
Place cakes on wire rack to cool for 10mins then turn out onto wire rack. Allow to cool completely.
Cream Cheese Frosting:
Beat butter and cream cheese until fluffy (2 mins). Add powdered sugar one cup at a time. Add vanilla and beat until fluffy (3 mins).
Place one layer of cake on a cake stand or serving plate. Top with 2/3 cup of frosting and spread evenly. Repeat with remaining layers and apply a thin coat of frosting all over the cake. Chill for 20mins.
Use the remaining frosting to frost the cake and do a rustic swirl on the sides and top with a large offset spatula if desired.
Decorate top with whole spices and gooseberries if desired.