You can never have enough shoes
I can clearly see the difference between EG and AEs while looking at them at Brooks Brothers. I would think that even the average person could tell if you put them side by side. But this is kind of like shipping to HDTVs at Best Buy...you can see the difference when they are together, but once they are on their own, most average people wouldn't know one from another.
I think some of the points made on this thread are:
1. Don't wear Stacy Adams shoes with a Canali suit and don't wear Edward Green shoes with an H&M suit, because the weakest link will show.
2. SuSu and Spier & Mackay, while less expensive than a GF BB suit, are not bad suits by any means. They are perhaps the AE of the suit world (as opposed to the Aldo) in that they are near the bottom end of the suit world when it come to being well constructed and nicely styled for the price.
Having said this, if you aren't comfortable wearing those suits with your shoes then perhaps put off the next shoe purchase and buyer a higher end suit. I don't think anyone else will notice or care unless you start buying cheaper suits or worse fitting suits.