urf
The legality, at least in part comes down to enforcability. The government, in keeping with usurping other freedoms justifies it by pointing out the number of degenerates who lost everything to a gambling addiction. They're doing it to protect us.
I don't know you, so please don't take this personally, but I think your voiced fears of how others might feel, if you informed them of your gambling, is your stuff. I tell people quite liberally, and most people find it interesting. They go on to ask questions, serious questions that suggest they are interested in trying it. I'm an MD and I have other doctor, lawyer professional friends and they too talk about it liberally.
I'm older (I was in the army during the Nam mess) and it might have to do with age, in that most of my friends are 40-60. At 20, I can remember telling friends that I was going to LV or Lake Tahoe to play poker all night and bet sports during the day, Saturday/Sunday.
You might get more responses if you take this to the "offshore forum/online sports betting" area.
Lastly, try telling a few people that you feel comfortable with and see how they respond. You might be surprised.