But to say that, whatever natural number you choose, that number will be non-negative, is to say something about all natural numbers. But it does not say anything explicit about the set N.
I'm not sure I understand what you are saying, but here goes:
"The set N" does not mean the same thing as "all N". Statements involving "all N" are statements about the individual natural numbers, to be translated as I suggested as "whatever n in N you may choose". However, statements involving "the set N" do not have to say anything about natural numbers. For instance, I can tell you that N belongs to the set {N}, without having to discuss any given natural number.
Cardinality is a property of a set defined in terms of mappings, which themselves are defined in terms of individual members of the set.