Residential energy constitutes a significant portion of the total US energy consumption. Several researchers proposed energy-aware solutions for houses, promising significant energy and cost savings. However, it is important to evaluate the outcomes of these methods on larger scale, with hundreds of houses. This paper presents a human-activity based residential energy modeling framework, that can create power demand profiles considering the characteristics of household members. It constructs a mathematical model to show the detailed relationships between human activities and house power consumption. It can be used to create various house profiles with different energy demand characteristics in a reproducible manner. Comparison with real data shows that our model captures the power demand differences between different family types and accurately follows the trends seen in real data. We also show a case study that evaluates voltage deviation in a neighborhood, which requires accurate estimation of the trends in power consumption.