A few things should be added to lay the foundation for further discussions. First gonochorism (the term to describe a sexual system where there are male and female members) does not always have to be linked to sexual dimorphism (the term to describe differences in appearance between male and females of a species). Sexual dimorphism is often a consequence of the respective reproductive strategies.
Among hermaphroditic species, one can actually also distinguish between various forms. The one OP is thinking about is considered simultaneous hermaphroditism, i.e. all individuals producing sperm and eggs, but there are also species who are sequential hermaphrodites. I.e. producing egg or sperm at different points in their life.
Studies trying to figure out fitness benefits have been investigating closely related species in which all three strategies are found, e.g. in certain worms. Here, it was found that the different species had different reproductive characteristics, that likely have benefits under different conditions. Generally, they found a trade-off between fecundity (how much they reproduce) and survival. Simultaneous hermaphrodites had the highest survival rate, but least fecundity (and smallest eggs, indicative of lower maternal investment), whereas the opposite was found for sequential hermaphrodites. The gonochoristic species was somewhere in-between.
Taking that all together (survival rate, reproduction over total life cycle etc.) it seemed that the dichoristic species had overall the highest fitness. They had higher fecundity in the early stages of life cycle. They outperform simultaneous hermaphrodites, which have lower fecundity. While sequential hermaphrodites are more fecund, they are delayed until their female phase, and during the whole life cycle they are not able to compensate the early advantage. Essentially they are able to reach sexual maturity faster, likely as they only need to produce one form of gametes. The disadvantage of that gonochoristic species pay is that they produce males, that cost the same as females (as eggs) but do not directly contribute to future generations (the limiting factors are the eggs). Hermaphroditism is speculated to be a primary advantage when population densities are low and it is difficult to find a mate.
There are also evolutionary developmental consideration. Transition from hermaphrodite to gonochoristic species is comparatively easy, as it could be reasonably executed by suppressing the development of one sexual function. Conversely, there are more steps involved in transition from gonochorism to hermaphroditism. I.e. once gonochorism outcompetes hermaphroditism in the evolutionary history of species, it is very unlikely that they hermaphroditism will develop, even if it became more advantageous.