Preheat oven to 350°F. Spray two regular sized muffin tins with cooking spray.
Whisk together flour, cocoa powders, baking soda, and salt. Set aside.
Beat butter and sugars on med-high until light and fluffy (approx. 2-3mins). Reduce speed and add eggs one at a time and vanilla. Beat until combined. Add flour mixture and mix until just combined.
Using a large cookie scoop (3 Tbsp) scoop dough into muffin tins, press down to flatten.
Bake for approx. 10-13mins or until mostly set, but still soft in the middle.
Remove from oven and immediately use a small jar or container to press firmly down in the center to create a well. Cool in pans for 15mins, then place on wire rack to cool completely.
Place milk and pudding into a medium bowl and beat for 2mins (I used a hand mixer). Place in fridge to set for 5 mins. Beat in pumpkin puree and pumpkin pie spice. Pipe or spoon into cookie cups.
Top with whipped cream if desired.
Note: I over-baked one of my batches of cookies and they were a bit too crisp for my liking (I prefer chewy). If this happens to you, just place the unfilled cookie cups in an airtight container with a slice of bread and leave to sit overnight. They will soften right up!