Using a stand mixer, cream the butter for 2-3 minutes, until light and fluffy.
Add in the sugar and mix for another 3 minutes, until fully incorporated.
Add in the milk, cornstarch, vanilla, and almond extract and beat on medium high until fully combined, scraping down the sides of the bowl as needed.
Sprinkle 1 cup of flour over the butter mixture, followed by the baking powder and salt. Mix on low until just combined.
Add in the remaining 1 ¼ cups of flour and mix again until the dough comes together. It may look a little crumbly in the bowl, but that’s fine!
Transfer to a parchment lined surface and top with another sheet of parchment paper before rolling out the dough to ¼ inch thick.
Cut out your desired shapes and place them about ½ - 1 inch apart on a lined baking sheet.
Repeat the process with the remaining dough.
Place the baking sheets with the cut out cookies in the freezer for 30 minutes. While the cookie dough chills, preheat your oven to 350°F.
Transfer the cookies directly from the freezer into the oven and bake for 8-10 minutes or until the edges of the cookies are golden brown. They will still look soft, but will continue to cook on the hot baking sheet!
Allow the cookies to cool on the tray for at least 20 minutes before transferring to a wire rack to cool completely.
When the cookies are cool, whisk together the icing ingredients. To create a marbled effect, drop 2-3 drops of food coloring into the icing and use a toothpick to create swirls around the surface.
Dip each cookie into the icing face down before lifting it straight back up and allowing the excess icing to drip back into the bowl. Place on a piece of parchment paper to set fully.
Allow the cookies to set for at least 30 minutes before stacking or eating. I usually leave mine out for at least two hours before we eat them!