I had to look on mine as I never noticed it before.
If it were me and I wanted to remove it, I'd get some matching putty stick to rub into the hole, that way it's reversible, no damage to the guitar.
Save the button in the case somewhere so it doesn't get misplaced.