I was having a similar problem with my outdrive not working right, just hearing the click of the switch and the drive not trimming up or down. Sometimes it would work fine, sometimes I'd have to keep playing with the switch, and it would usually go up or down eventually. I bought a replacement trim switch and when I remomved the switch to see about replacing it, it started working a lot better (but still not perfect) so I left the old switch on and kept using it. This season it started to get worse and one day at the marina couldn't get it to go up or down at all no matter what, so I had the new trim switch installed and its been working perfectly since.
I ordered the switch from this shop and they were pretty helpful in getting back to me letting me know that I was ordering the correct replacement part.
http://maxrules.com/fixomctrim.html