Many already believe it is time to end Affirmative Action as it is unfair to those who have never wronged anyone, and gives an advantage to those who have not been wronged. And to clarify that point, when I say "have not been wronged", I am speaking to the specific correction made by AA. For example, if a black person is applying to University for the first time, they have not been wronged by the application process. So it can be reasonably argued that giving someone preferential treatment in college admissions is as unfair to whites as Jim Crow was unfair to blacks. Instead you should just make the admissions process fair for all.
I personally support making up for past mistakes, such as by the use of Affirmative Action, but I tend to be hopelessly pragmatic. In my mind, giving blacks something of value not given to whites (money, admission preference, etc.) is too big a bite at this time in our history and is doomed to failure. I therefore think we have a much better chance of meaningful change if our goal is equality for all.