We use Tryptic Soy Broth (TSB) for the same purposes and we have seen the same thing.
What process is the TSB involved in? Is it coming in contact with any chemicals? Is there any temperature ranges the TSB is maintained at? Is there any mechanical shock being subjected on the media (vigorous mixing)? Is there a pH change in media that has the fiber-like particulate? Have you run any controlled experiments to try and reproduce the particulate?
Any ideas I have had so far on why this would be happening have been invalidated by lab experiments.