The best place to start is locally - your state or city may offer incentives for you to locate there. You may be able to get tax credits or other incentives for locating in depressed areas of your state or county. Check with your state government and also with your city and county agencies.
You should also check with any nearby community colleges or universities to see if they have a small business development department (or some similar name) that offers free services to startups. As an example, Murray State University near our office has an excellent center with some services comparative to what we provide in my CPA firm, but at no cost.
A CPA would be helpful in working out the tax angles for you in other ways - choice of entity, when and how to depreciate your assets, deducting your startup costs, putting profits in a retirement plan, and more, including setting up your bookkeeping system and reading reports.
I typically advise business startups to have a year of living expenses set aside before quitting a job to work in the new business full time. If you are able to continue in your day job, just don't plan on getting much sleep for the next year!
Good luck with your venture and I hope this helps!