Never Trash the Folder,
After downloading your content you should file or backup the download you wish to keep. If they where Apps they should now be installed into the Applications Folder, if other media then filed in the right Library or Folder. Once that is done then move the unwanted Files to the Trash.
I work on the web a lot and currently only have 5 things in my Downloads, 2 PDFs waiting to be moved to my Documents Folder, 1 Music Software waiting to be installed on my Laptop and 2 Software Installs that have been done and I need to back them up before deleting.
Its best to keep on top of this workflow...