Residential energy consumption accounts for 22% of the total energy consumption in the US. However, the impacts of local planning policies, such as increasing density and changing the housing type mix, on residential energy consumption are not well understood. Using Residential Energy Consumption Survey Data from the Energy Information Administration, quantile regression analysis was used to tease out the effects of various factors on entire distribution on the energy consumption spectrum instead of focusing on the conditional average. Results show that while housing size matters for space conditioning, housing type has a more nuanced impact. Self-reported neighborhood density does not seem to have any impact on energy use. Furthermore, the effects of these factors at the tails of the energy use distribution are substantially different than the average, in some cases differing by a factor of six. Some, not all, types of multifamily housing offer almost as much savings as reduction in housing area by 100 m2, compared to single family houses.