Some of the greatest exponents of Islam were Sufis or associated with Sufi Orders. These include Ghazali, Imam Hanafi, Ibn El Arabi, Ali Hujwiri, Jalaladin Rumi, Abdul Qadir Gilani, and more.
Just because there are cults, deviations, and other distortions doesn't mean the whole 'craft' so to speak is deviant. Remember Rumi's words "There is false gold only because there is such a thing as the Real..."