We have had this analogy many times over. If you belong to one fraternity in college, and you go to another one, and while you are there, talk crap about your old fraternity, and then later on you decide you want to come back, you will find it very hard to do so.
As for the IFBB, each individual decides if he wants to compete there or not. As long as they are in good standing with the organization. They choose to compete, they don't have to.