Ant uses basedir as the project's main directory and thus expects almost everything to be relative to that directory. Setting the basdir to the src directory is probably going to mess things up. But it all really depends on your project's directory structure. A typical project should have a structure such as:
project/src/** (your Java sources go here)
project/bin/** (your compiled Java classes go here)
and so on. Thus the build.xml is in the base directory of the project (basedir=".").