Thanks for the responses!
As per Atheist's comment, I think I'm a lot more interested in what you call "real" math - I already have enough applied math to deal with as an engineer. I've been studying discrete mathematics (used for signal processing and such) lately, and even that is a bit too "applied" for my recreational tastes.
So I think I'll start looking into group theory (is this the same as set theory, of which I occasionally hear mention?) and maybe some topology.
Also, lurflurf's suggestion of "algebra or calculus" intrigues me. What sort of things do more advanced levels of algebra and calculus cover? How are these different from the precalculus algebra and differential/integral calculus I've taken?
Again, thanks!