Use the -Xmx command line switch. Specifically for Eclipse, do this:
eclipse.exe -vmargs -Xmx512M
This gives Eclipse a maximum of 512 MB memory.
Note: I am assuming you want to give the Eclipse IDE itself more memory. If this isn't an issue with the IDE itself but with your own program which you are launching from the IDE, then you'll need to do something different: Select Run / Open Run Dialog, then select the configuration for launching your application. Click the Arguments tab, and in the box "VM arguments" add something like "-Xmx256M" to give your application 256 MB of memory.