The golden rule is a oversimplification of Kants categorical imperative which is

“Act only according to that maxim whereby you can at the same time will that it should become a universal law.”

So it actually accounts for people being different by saying it has to be applicable to everyone (universal law).