Please UseCodeTags(<-click). I've added those for this time.
Here's the thumb rule for modifying the HashMap while iterating it:
When a HashMap is being iterated by an iterator, then any addition/removal of an element must be done via iterator itself.
Currently, you are using an iterator(returned by context.keySet()) to iterate over HashMap, but using methods of HashMap to remove the element from HashMap.
If you look carefully, the ConcurrentModificationException is thrown at line which iterates over HashMap (the for loop in your code), and not at the line which removes an element from HashMap.
Also, in second piece of code, what you are actually doing is: modifying the element Key_2 (and ConcurrentModificationException is thrown only during addition/removal). Try adding a new key/value pair (e.g. context.put("key_3","value_3")) and you should get ConcurrentModificationException.
To fix this, there are two approaches:
1) Use iterator's methods to add/remove from HashMap. e.g.
2) Use ConcurrentHashMap instead of HashMap. In this case, you don't need to follow the iterator rule (that is, your current code will work without throwing ConcurrentModificationException).