If you un-check an already installed plugin, hit Make it so! and open plugins dialog again, the plugin will still be "checked". This is just a visual bug, the plugin is already disabled. This issue can actually be very annoying if you want to switch between two plugins like I did. After I disabled the first plugin, I opened the plugins dialog again, activated the second plugin and hit Make it so! to confirm the changes. But because the initial plugin (which was disabled) was automatically checked because of this visual bug, it activated both of the plugins, not just the one I wanted to activate.