You have to understand something about the way computer store floating point values.
A double does not have infinite precision, so some numbers cannot be represented 100% accurately. The following article explains it in detail, but it's a bit technical:
What Every Computer Scientist Should Know About Floating-Point Arithmetic.
If you really need infinite precision, consider using java.math.BigDecimal to store your numbers. In particular if your software is working with amounts of money,
you should use this, because rounding errors are most of the time unacceptable, even if it is just about a few cents.
Otherwise, you can use something like String.format(...) to format your numbers properly before displaying them on screen.