I ditched all but 4 of them. Same with my kitchen towels. I try very hard, not always successfully, to constantly edit. Our house is small because I prefer it that way and as a result we simply do not have the room for crap.
How many pot holders do you have?