Although they do nothing once you delete the associated application, they do take up some disk space. Applications may create preference files that are stored in the /Home/Library/Preferences/ folder.
Most OS X applications are completely self-contained 'packages' that can be uninstalled by simply dragging the application to the Trash.
