It depends on what you are creating. For e.g., if you are creating a web app in JSP/Servlets (say for monitoring a domain), JMX is the obvious choice. Just include the wlfullclient.jar in the classpath and you are good to go.
On the other hand, if all you need is to quickly automate administration stuff like deploying apps, monitoring server health, generating alerts, stop/start servers etc then
you should use WLST as it is quick and simple.
Performance wise there is hardly any difference in overhead. It would rarely be a factor in deciding between the two.