Chickens can and do get excessively cold during the winter. You want to make sure they have shelter from the weather and wind. You want to make sure they have a place to sleep that's off the bare ground. You want to make sure they have plenty of straw for bedding on the coop floor and in their nesting boxes. And you want to make sure they have plenty of food and access to unfrozen water. Here's some more information about specific winter-related topics.
Signs of cold: If when you come in the coop in the morning, the chickens are shivering or don't seem to be thriving, it's likely too cold for them. If you add a heat source (see below) and you see evidence that they huddle under it, that's a sign it was too cold and you were right to add the heat source. Also, if you've got chickens who aren't in great shape and/or are missing substantial feathers, as happens in rescue situations, you'll likely want to provide some heat
Heat option if not miserably cold: In the High Sierra where we're located, it only gets in the single digits for a few days each winter so the cold isn't too bad. But we still needed something. We got something called a Hound Heater that's for dog houses. It's basically a lightbulb in a vented metal box. It's attached on the coop wall with a cord running outside to an outlet and a Thermocube. The cube is just a little prong adapter that switches on when it gets below 32 degrees and stays on until it reaches 45 degrees.
Heat option in really cold climates: Brooder lights are helpful. You want to use ceramic heating elements, not light bulbs in them, though. They're kind of expensive, but they work well, according to Farm Sanctuary in New York.
Vaseline on combs? I asked Susie at the New York Farm Sanctuary, which gets really cold, if she's ever heard of using Vaseline on the combs to prevent frostbite. The way I read her response is that it might work, but that providing a heat source is the best course of action. Here's her reply: "This [Vaseline] can work but not always — and frostbite does occur, especially with older birds, and if they have circulation issues. Also birds with really large combs are more prone to it. We get so many birds who have lost feet, toes, combs, etc. brought to us after a hard winter."
Insulating the coop? Insulating your chicken coop is helpful but can lead to rodent issues. Rodents like to live in insulated barn walls, and rats can and will kill a chicken. So make sure you rodent-proof the coop as well.
Frozen water: We aren't too fancy at our place. We give the chickens fresh water that we lug in jugs from our house each day and night. They need unfrozen water. One thing that's been recommended to us and that we're looking at for next year, if we can afford it, is one or two water dispensers with a heating element underneath that needs to be plugged into a socket. They are available many places, but the specific ones recommended to us were these Brower Co. units.