I'm prepending the observation that Advan046's observations make it sound all the more likely that stuff is just b0rken and "correct behavior" may not be achievable without some of that demolition, but as long as I've written it...
EDIT: PreS (like a PostScript but before); I just saw an article on working around the inability to break inheritance because of the number of items in a Library. I'm suddenly nauseated over the sheer opportunity for weird behavior in SP, and am mostly assuming that the below is not useful because SP is just doing something awful that has nothing to do with the logic of permissions...
What's that quote? "I'd have written you a shorter letter but I didn't have time." TL;DR for the below: I still don't see where separate groups are being created. Re-reading the initial post, it still just sounds like the Document Library goes from saying "let whoever has access to the Site in" to "let the group who had access to the Site when we broke inheritance in." And that's just the same group, referenced from both places.
The loooong version. So many apologies:
Sorry it wasn't more directly applicable; I just re-read your initial post. I'm confused because it doesn't sound to me like you're creating a new Group, and I don't see an operation which would create a separate Group for the Document Library (unless this is one of those things that's intrinsic to Doc Library inheritance breaking, but the idea of generating random new groups automatically gives me the heebie jeebies). I don't know what the text is that you're referencing that's being contradictory about what to expect, but what you're explaining is consistent to me with the notion of Groups in a general access and permissions way (it's a concept that comes up a lot regardless of platform).
It's entirely possible that I'm failing to understand something (make that "probable" and "a lot of things"), but it looks to me like you've got Group A that has permissions for the Site, and from the Site inherits permissions to the Library (before you break inheritance). When you break inheritance, you now have independently the facts that Group A has permission to the Site, and that Group A has permissions to the Library because it got its own copy of the permissions it had at the time of breaking inheritance, but Group A is still Group A, and if you add someone to Group A, they show up in both places because it is still just the one Group. That is, they're not "still connected," they're actually the same group. It's just that in the Library it's got its own copy of the directive to "let Group A in" instead of inheriting "let whoever can get into the Site in."
You can remove Group A's permission to the Library, and you can add Group B's permission to the Library and then only Group B would have access to the Library. You could add Group C to the Site and it would not automatically get access to the Library because you've broken inheritance. Users within those groups will always have the permissions of the group, and can be added or removed from the groups and will gain and lose the group permissions when that happens.
I have the terrible feeling that I'm trying hard to explain something you already know while I'm missing the part that means I've missed the point...
I also realize I sounded pretty dismissive of working with SP, and that was sloppy, inaccurate, and inappropriate. Wrestling with stuff like SP is miserable, and I think it's a rough gig to get stuck with all the "real" software problems and only SP to fix them with. It seems to be a terrible tool, and my aggravation is just with the business notion that you can just get a tool that doesn't require "writing code" and it can just be a part of another job. I tend to dislike frameworks because you get stuck having just as much trouble learning them as you would a programming language, and all the complexity that's supposed to make things easy invariably ends up making unusual use cases harder and problems harder to find and fix. In any case, my respect and heartfelt sympathies for folks stuck wrangling SP.