That's going to be hard, and I doubt it can be done automatically.
You first would have to find all Properties objects that load from that properties file (already quite hard - sometimes the properties file can be relative, otherwise absolute). Then you have to find all calls to getProperty on those objects and retrieve the property name from those calls. And that for all applications that can potentially use the file.
Now, if it's just one application, and one point of reading the file, you can choose to rebuild the Properties objects at that point. Strip away all properties you don't plan on using:
Afterwards the Properties object only contains the properties you are using (provided isUsedKeyForMyApplication is implemented properly), and when you store it again the file will only contain the properties you want.
Of course it is a LOT more difficult to do if there really are multiple applications reading the file, or multiple points at which the file is read (although the latter can be handled by a single method call).
There's no way of really ever knowing this. All you can do is scan your source code to see what properties they reference. Mind you there are binaries/programs that could be referencing properties and you wouldn't necessarily know it. But I assume you are talking about properties for your application only here.