This week's book giveaway is in the Mac OS forum. We're giving away four copies of a choice of "Take Control of Upgrading to Yosemite" or "Take Control of Automating Your Mac" and have Joe Kissell on-line! See this thread for details.
Deleting is very picky. If you tell it to include or exclude specific files, then it deletes only files. If you know which directory contains the *.cache file, you can specify that instead. For example, assume all of the *.cache files are in a directory named temp. Then the delete would look like:
(By the way, your last sentence doesn't match the delete command you gave, so I assumed that the sentence was correct and the delete command contained a typo)
If you are new to Ant and plan to use it for a while, I really recommend you get a book such as Ant in Action. It is really good about explaining all of the intricacies and contains a lot of helpful example projects.