There is an app for xbox called "Xbox" which supercedes Smart Glass".
In the settings from your xbox you can specific who can connect to the xbox via Xbox app.
1 Press the Xbox button to open the guide.
2 Select Settings and then select All Settings.
3 Select Preferences and then select Xbox app connections.
4 In Xbox app connections, select the option that best meets your needs