That's the naive approach, which is broken because of daylight savings - there are two days a year that are not 24 hours.

The right approach indeed uses Calendar.get() in combination with both Calendar.DAY_OF_YEAR and Calendar.YEAR. In short:
- subtract the day of year of the first date from the day of year of the second date
- add the number of days for each year in difference as well; use Calendar.getActualMaximum(Calendar.DAY_OF_YEAR) to determine how many days a year has.

To make it easier, you can swap the operands if the first date is larger than the second: