Does a hardware wallet really need to “store” your seed phrase?

The concept of stateless wallets (devices that don't store the secret permanently) is gaining traction. Would you feel safer knowing your device is 'empty' after you turn it off?