What can you do if you put too much water in your pot of soup?

I am making soup and I am pretty sure its way too watery. Can I add something to it to thicken it or should I just dump out some water (might be hard, since its in a crock pot)?