Despite how far in the past you look, the color shift will still tell you if it accelerating and decelerating, and as for the distortions when you deal with an object as big as a galaxy it's very hard to distort all of the light, maybe small amounts of it but not all of it and thus you would still see the red shift. The only thing I could think of that would distort a whole galaxy would be a blackhole about halfway between it and the observer. Even then the blackhole would be moving so the distortion would not last long.