Well, I'm following Oxford American and making it a plural possessive, "wits'", since we would say, "I'm at the end of my wits" not "at the end of my wit" (which could well mean I can't make jokes any more). So this baby's getting listed in my "Problem words" as "at one's wits' end."
I'd say, after the "s" -- presumably one has multiple wits. The dictionary says that "wits" should be plural when referring to one's "powers of intelligent observation, keen perception, ingenious contrivance, or the like." However, the dictionary also puts the apostrophe before the "s" in this particular phrase. Still, the dictionary has been wrong before... ;-)
Edit: On the dictionary page for end, definition #33, this phrase is listed with apostrophes in either place (dictionary.com, anyway).