I am using Apache,MySQL,Java ... On the client machine i cannot install all this software one by one ..
So is there any software that will allow me to integrate all of this into one exe file ..
So that the client can just click that installation file and all necessary things will be installed
There are one-click installers for the *AMP stack (Apache, MySQL, PHP/Python) e.g. XAMPP.
But you need to be careful that you set up all the security/admin stuff correctly: if you tell your customers "Click this to install everything" but your process breaks their systems or lets the whole world access their confidential information, then you may be liable for some expensive repairs.
You also need to make sure that you do not bundle your software too closely with open source software under the Gnu Public Licence without considering the licencing implications.
As for Java libraries etc, I don't know, but you might want to look at tools like Maven to set up a standard build mechanism.