If at all possible never pay your payroll taxes late.  If you have a choice of making the payroll tax deposit and an estimated quarterly payment, chose the payroll every time.  The penalty on unpaid payroll taxes starts at 50% of the unpaid amount and can be as high as 100%.  The reason behind this is that the payroll taxes are money you withhold from your employee in trust for the IRS.  So if you don't file and pay that money in, it is not just an underpayment of taxes, it's actually theft.