What do you mean, access Map.Entry? Do you mean that you iterate over the entrySet() each time? That's how it works.
There are actually three views you can use: keySet() for only the keys, values() for only the values and entrySet() for both. With entrySet() you need to have something that also contains both, and that's Map.Entry. That's why the iterator returns those.