The pyramid construction project was fueled by Egypt's many landed estates, each of which was required by royal decree to supply not only workmen, but draught animals, food, and produce for the project. In addition, there were hundreds of "pyramid estates" established throughout Egypt during the Fourth Dynasty for the sole purpose of providing food, produce, draught animals etc...to the pyramid construction sites. Read Mark Lehner's "The Complete Pyramids." Somewhat off topic, but it has also been proposed that the Great Pyramid may in fact have built Egypt. The establishment of hundreds of "pyramid estates" likely led to the colonization of much of the Nile Valley. These estates formed the nucleus for many of towns and later urban settlements that dotted the Nile Valley.
And your math is incorrect. Houdin is every bit as competent an engineer as you, and he has confirmed the twenty year estimate.