Cleanly shut down PeerManager by disconnecting before we quit