What do you do with the spare buttons that come with new clothing?  Try this:

Designate a box for them and keep a fine-point indelible ink pen inside. When you buy a new item of clothing with a spare button, which usually comes in a small plastic or paper bag, write a brief description on the bag (to identify it when you need it) and put it in the box. When the box fills up, sort through it and throw out buttons for clothing you no longer own.