First hit on a google search is this page. Of course, you should probably check the documentation of your particular distribution as there may be distribution-specific procedures. Are you installing the client or the server?
On a Red Hat or Fedora system, I don't recommend simply deleting directories. You may leave forgotten bits and pieces behind, including entries in the RPM database. Not as big a problem as yanking out a Windows app by brute force, but still, it's untidy. And, since the RPM version puts files in the /usr/bin/directory, you have to be careful to delete only the CVS files.
The command "rpm --erase cvs" will remove the cvs package. You might want to check to see if also deletes the cvs user and group IDs. A lot of the uninstallers leave the local config information in case you change your mind or something.
An IDE is no substitute for an Intelligent Developer.