And if it's a directory you will need to remove all files and sub directories in it first.
Also, if you have opened an InputStream / OutputStream / Reader / Writer to the file and haven't closed it (which you should do in a finally block, or preferably in Java 7 using try-with-resources) that will also prevent the file from being deleted.
I was trying to delete those files because it is saved in the database as a binary format. Every time any view is made for the image files creates a .JPG file in that folder. So, these are being a dump amplifying application size.