Hi, as charon is saying, this i easier said than done, mainly because the differences aren't black and white - cancerous and normal cells are still very similar, and the differences are gradual in nature.
There are, however, a few compounds that have been used for this, because they seem to be doing stuff to cancer cells while leaving normal cells alone. Butyric acid kills colon cancer cells, but increases viability of normal colonocytes, sulphoraphane (from brocolli) kills cancer cells, lycopene (tomatoes) appear to be doing something to prostate cancer cells and on and on we go. Most of them, i imagine, work by modifiying the structure of DNA so that 'anti-cancer' genes can do their work (Histone deacetylase inhibition if that rings a bell), which means they target one of the differences between normal and cancer cells.
Obviously there are many other routes of action: differential expression of surface proteins, elevated metabolism, increased vascularity in/around tumor and so on. Many, many billions are being used by many, many different researchers right now.
A nice way to detect how compounds act on cancer cells, is simply just to incubate them with the compound and see how they survive. Include normal cells if possible, and make sure concentrations are realistic. Some compounds work in mysterious ways - some antioxidants, i imagine, will produce toxic hydrogen peroxide by reaction with iron in the media and then there isn't really anything magical about the compound.