Leaving groups and basicity
The key factor in determining a species' suitability as leaving group is its basicity. In general, the weaker the base, the better it is as a leaving group. To understand why this is so, consider the reaction below:
The molecule on the left is called toluenesulfonic acid - or just tosylic acid. It is a strong acid. Strong acids dissociate completely to give the corresponding base and a proton, which is the reaction shown above. They do so because the negative charge is well stabilized in the corresponding base. And the corresponding base of a strong acid is a weak base.
In other words: The stronger the acid, the more stable the corresponding base when acting as a leaving group from the proton. And that is exactly the same as when acting as a leaving group from another molecule. So when considering how good a leaving group something is, think about its corresponding acid: The stronger the corresponding acid, the better the leaving group.