the discipline
Probably the number one challenge to a programmer starting a business is maintaining your discipline. Programmers are naturally creative people. Maybe not in the art sense, but in terms of always wanting to create new things and explore new ideas. This aspect of the programmers makeup is simultaneously our greatest strength and our greatest weakness.
If you're going to start a company you need to be able to focus 100% of your creative energy into your product and marketing. There's simply no time to be distracted with other interesting ideas. Once committed to developing a product it's so important to stay on track. It's probably the hardest part of getting things rolling. I've struggled with this from the very beginning.
Since starting HelpSpot I've installed at least 3 wiki's with ideas for ISV oriented websites, started coding a PHP system for handling directories of information at least 3 times (i need this for something else I shouldn't be working on!), started a stock oriented blog, and wrote up some plans for another software product. I'm happy to report that none of these lasted more than a few hours before I kicked myself and stopped working on them. Any could have spiraled out of control and lasted several weeks before I realized I was off track.
I don't have any great techniques for preventing these "attacks" of creativity, but I do find that channelling that energy someplace else can help. For instance, I often write a blog post when I find my mind wandering off to new programming ideas. This gives me a creative release that's also a worthwhile thing to spend time on.