I noticed that no one responded to your question, and since I haven't been on here in quite some time, this seems like as good a place to start as any.
In my humble opinion, it seems like you found the answer. It doesn't say "Personnel will not DISPLAY security identification
badges outside the area for which they are required", it says that they will not WEAR them. I believe that includes whatever is holding them as well.
As far as I can tell, you're not going to find anything in writing to fully support your argument. However, common sense should tell a Soldier that:
1. Just wearing the holder on your uniform is a no-go.
2. It's not an award, insignia, or accouterment authorized for wear.
3. If you can stuff it in your pocket, you can unclip the whole thing and stuff it in there too.
There may be some precedent shown in AR 670-1 Para 3-6, 2b, where it mentions security cords or chains for cell phones. They must be hidden from view. I would believe the same would be true for security badge holders not in use and outside of the area they are meant for. It detracts from the professionalism of the uniform.
Sorry I don't have anything set in stone for you.