Dude organic gardening is cool. The way I do mine is effectively organic... but yea, I always have issues with the bugs too, especially corn around here.
There are ways to battle the bugs... like using vinegar to repel worms, coffee grounds, etc.
Where you're at there probably isn't all that much else to eat, so a pesticide-free garden is a little slice of heaven for creatures.