I'd have to look at your applications folder, for example. A user Library contains a lot of files used by applications, such as preferences, application support, fonts, widgets, scripts, etc. The number does seem high, but who knows what apps you have. Take a look at your Library and sort it by size. Then check what the largest files are.
One thing you can probably do is to run a good maintenance program. For example:
MainMenu,
Cocktail,
MacJanitor. Eliminating font and browser caches could clean up unnecessary files in your library.