Yes. Doing so allows you to restart the server and keep all of your session data. It will also be necessar if you want to run your app in a clustered environment with session replication.
Do you know how servlet container (any you know) behaves when some session entries can't be stored. Does it abort entire process or just skip "bad" entries?
I am not sure about all App servers. But Weblogic (atleast 8.1 and above) skips the bad entries as you had mentioned. It also gives a log on the console saying something like 'non serializable content present in session'.