Incompatibility problems when altering serialized objects

[Kari Nordmann]

I'm working on a hobby project that is alot bigger than my previous work, and the complexity of the program is teaching me alot.
I have now met an issue that I don't see any easy way to pass, and I would like to know how this works in the world of professional programming.

Alittle on my program:
My program has several object classes that make out a core of the program.
One object class is Object_Gear. Object_Gear is dependant on datas from objects of type Object_Stats, Object_DropSpot, Object_Buff and Object_Gem, in addition to normal variables. Object_Stats is an especially crucial class, few things will work properly without it.
All instances of Object_Gear, most instances of Object_Stats and some instances of Object_DropSpot will be created by the user. The user will build up a database, that makes up alot of the point of the program.

Although I have planned alot on my program, I am not entirely sure of all I need from every object class. It is a program made for a complex task with alot of details, and I find it hard to plan out every little thing. The task of the program might also change slightly with time, it is something I can't control. Thus my program has to be compatible with changes in the object classes.

The problem:
Right now I use readobject\writeobject to save and load the database on a *.data file.
Whenever I do a change in any object class, all previously saved instances of that class are incompatible and cant be loaded. The database is lost.
How am I supposed to be able to do changes to an object class and still be able to fit in all instances of the old, now non-existant class?
If I add a variable to one of the classes, how can I tell the program what variables we do have in the file and what has to be filled in, to make it fit?
If I remove a variable in a class, how can I tell the program what variable from the file should not be filled in, and which should?
How is this usually done in larger systems?

[Rob Spoor]

Originally posted by Kari Nordmann:
Right now I use readobject\writeobject to save and load the database on a *.data file.
Whenever I do a change in any object class, all previously saved instances of that class are incompatible and cant be loaded. The database is lost.
How am I supposed to be able to do changes to an object class and still be able to fit in all instances of the old, now non-existant class?
If I add a variable to one of the classes, how can I tell the program what variables we do have in the file and what has to be filled in, to make it fit?
If I remove a variable in a class, how can I tell the program what variable from the file should not be filled in, and which should?
How is this usually done in larger systems?

First of all, define a special field in your class:

private static final long serialVersionUID = XXXX;

If you omit this, a default value will be used. If you change the class signature (e.g. add or remove fields) a different value will be used.

If you keep this value the same, you will find that there will be no more "incompatible class version" errors.

That alone will already be enough if the fields will remain the same - the same names, types and order. If you change this too, you will need readObject and writeObject. Fortunately, you're already doing that.

Now if you add or remove fields that will be harder. Although there is one quite easy solution - use a Map with the field names as the keys and the field values as the values (boxed if needed). If a different version doesn't care about a field, it can ignore it. If it misses a field, give it a default value.

In code:

import java.io.*;
import java.util.*;
 
public class Test implements Serializable
{
    private static final long serialVersionUID = 13;
 
    private int field1 = 5;
 
    private String field2 = "Hello World";
 
    private Object[] field3 = {field2};
 
    public String toString()
    {
        return "field1 = " + field1 + ", field2 = " + field2 + ", field3 = " + field3;
    }
 
    private void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException
    {
        Map<?,?> map = (Map)in.readObject();
        Integer i = (Integer)map.get("field1");
        field1 = (i == null ? 0 : i);
        field2 = (String)map.get("field2");
        field3 = (Object[])map.get("field3");
    }
 
    private void writeObject(ObjectOutputStream out) throws IOException
    {
        Map<String,Object> fields = new HashMap<String,Object>();
        fields.put("field1", field1);
        fields.put("field2", field2);
        fields.put("field3", field3);
        out.writeObject(fields);
    }
 
    public static void main(String[] args) throws Exception
    {
        Test test = new Test();
        System.out.println(test);
        ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream("file"));
        oos.writeObject(test);
        oos.flush();
        oos.close();
    }
}
 
import java.io.*;
import java.util.*;
 
public class Test implements Serializable
{
    private static final long serialVersionUID = 13;
 
    private int field1;
 
    private String field2;
 
    private Number field4;
 
    public String toString()
    {
        return "field1 = " + field1 + ", field2 = " + field2 + ", field4 = " + field4;
    }
 
    private void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException
    {
        Map<?,?> map = (Map)in.readObject();
        Integer i = (Integer)map.get("field1");
        field1 = (i == null ? 0 : i);
        field2 = (String)map.get("field2");
        field4 = (Number)map.get("field4");;
    }
 
    private void writeObject(ObjectOutputStream out) throws IOException
    {
        Map<String,Object> fields = new HashMap<String,Object>();
        fields.put("field1", field1);
        fields.put("field2", field2);
        fields.put("field4", field4);
        out.writeObject(fields);
    }
 
    public static void main(String[] args) throws Exception
    {
        ObjectInputStream ois = new ObjectInputStream(new FileInputStream("file"));
        Test test = (Test)ois.readObject();
        ois.close();
        System.out.println(test);
    }
}

As you can see, two different classes with the same name and serialVersionUID, but different fields.

The output (writing first, reading second):

field1 = 5, field2 = Hello World, field3 = [Ljava.lang.Object;@3e25a5
 
field1 = 5, field2 = Hello World, field4 = null

As you can see, field1 and field2 are deserialized just perfectly. field3 is ignored, while field4 gets a default value (null).


Note that it is VERY important that you check for null for all fields, and preferrably the type too.

If you want to keep any ignored values as well, you can also make the Map a private field; you assign it the read value, and only overwrite the values when writing the Object.

[Kari Nordmann]

Ah thanks alot