1. If you change the recipe of black powder it is not considered black powder any more.
2. Amonium Nitrate is hygroscopic and thus do not tolerate moist air leading to problems with ignition and corrosion due to absorbed moisture.
3. Because it is not stable, it changes crystal structure depending on temperature and this means you have to store and use the AN based powder at once ore store the material at a controlled and constant temperature to avoid dangerous and uncontrolled cracking of the grains.
AN based "black powder" have been tried and used, the temperature dependent phase shift in crystal lace made the big grains of powder reduce to fine powder over time making the ballistic properties change quite a bit.
A big block of BP burns quite slow but if you cut the block in smaller peaces increase the surface area it will burn explosively. So what happened was that old AN based "black powder" made the weapons explode in a dangerous way.
AN is sometimes used in experimental rocket fuels due to low cost and no solids upon combustion, but to avoid catastrophic engine failure the AN needs to be phase stabilized with additives to reduce or hopefully eliminate the cracking of the fuel grain.