Check with your accountant, your loan officer, and your fund administrator, but in many cases you can withdraw certain amounts of money penalty free from retirement accounts if the money is used for the purchase of a primary residence. In some cases, you can even borrow money against these accounts and pay six or seven percent interest on the money you’ve borrowed back into your own retirement accounts.