I encrypt everything that leaves my house since it could be easily lost or stolen, but it is rather inconvenient.
If someone breaks into my house, I’ve got bigger problems than someone getting their hands on my media collection. I think it would be more likely for me to mess something up and loose access to my data than for someone to steal it.
Just keep in mind that you can’t wipe the bad sectors that have been remapped. That’s unlikely to be an issue for a personal drive, but something to consider if it held particularly sensitive information.