Behind the scenes, the javafxpackager just shells out platform dependent open source installer creator products (e.g. rpmbuild for redhat type linuxes, dpkg for debian type linuxes, wix or innosetup for windows installers, osx app or dmg creator for mac, etc).
The UI (if any) for each of these generated installers is all specific to the installer and the OS it is being generated on.
The installer UI can be customized by providing resources and custom configuration files to the underlying installer creator products that the JavaFX packager shells out to.
The amount of installer UI customization possible depends on the underlying tool. For example a windows WIX installer is very customizable, and an rpm package has no UI at all as the UI for the rpm install is completely handled by OS toolkit extensions the user uses to install the rpm.
None of the installer UI is written in JavaFX, it is all native to the platform.
Honestly, if you are going to have somebody install your software, then I think it is a good thing just to have a standard UI for the software install that looks like any other software installer for that platform. Think on iOS for example, where everything is installed or updated in exactly the same way - it's simple and intuitive for the user - why confuse the user installing your product before they even get to use it? Unless the installation takes a long time (like a large game or office), I think, in general, custom installers are more hassle than benefit to the user - even then the cross sell messages that often show up in custom installer screens annoy me.
Also if you want to eventually deploy your app to an app store like the Windows store or Mac App store (something that near future releases of the javafx packager will greatly assist with preparing for), you won't be able to use your beautiful custom installer anyway, because the store terms of service will demand that you use the store's standard software packaging and installation mechanism.
My advice is to spend your effort elsewhere (e.g. in making your app itself better) and just use the out of the box packaging provided by the javafx packager, with a few custom tweaks for icons, license pages, etc. Personally the effort involved in just getting that much to work with the standard installers, is more than I would normally want to spend on an installer development task anyway.
One other option is that you can use WebStart with a JavaFX coded Preloader.
I don't necessarily recommend that over a native installer, especially seeing as you have Linus as an OS target and WebStart won't function out of the box with an OpenJDK install on a Linux box (or a Linus either ;-).
WebStart is a reasonable alternative for Windows and Mac.