Wearing a down jacket in bed is a mistake, as you loose all the fluffyness squashed indside a sleeping bag, even worse if the bag is squashed by another bag!

If course having more insulation is good, but my theory about wearing too much to bed is based on my experience of wearing too much to start with (feel cold getting into a not-yet-warm bag). After a while the bag warms up. I fall asleep, start sweating, night gets colder, I wake up wet and freezing with all my clothes on at 3 am...

Eat sugery food before bed time, exercise a bit to warm up, wear just a bit too little to start with, make sure the matress is good enough (use spare fleece underneeth), wear even layer of clothing with no cold spots like feet or head.

By the way, layering is not warmer with clothing either (warmt/weight), it is just so convenient with widely warying temperatures and activity states that is is taken for granted, and we need to take also wind and rain into account when considering which clothing we take with us.