Preheat your oven to 350°F (175°C) and line a muffin tin with paper liners.
In a large bowl, whisk together the flour, sugar, baking powder, salt, and ground cinnamon.
In another bowl, mix the melted butter, milk, eggs, and vanilla extract until well combined.
Pour the wet ingredients into the dry ingredients and stir until just combined.
In a small bowl, mix the brown sugar and ground cinnamon for the filling.
Fill each muffin cup halfway with batter, sprinkle a layer of the cinnamon filling, then top with more batter until the cups are about 3/4 full.
Bake for 18-20 minutes or until a toothpick inserted in the center comes out clean.
While the muffins are baking, prepare the glaze by mixing powdered sugar and milk until smooth.
Once the muffins are done, let them cool for a few minutes before drizzling with the glaze.