This reverts commit aee3b50212. Turns out this makes it so that I get instant wakeup after sleep
aee3b50212