Hardest thing you found about starting up ?

I'm interested in what you guys and girls found the hardest part when starting up your business or what it was that stopped you from starting a business around an idea you had ? I personally keep struggling with the coding side of things like a lot of others i think but i'm intrigued in what everyone else has found the hardest bit about starting.