Just because someone unvotes doesn't mean they are no longer suspicious of them. It could be they are reading up on information, going away for a while and don't want to be involved in a lynch they didn't see, or they have someone else in mind that is more suspicious.