As you probably know by now, Demi Moore has filed for divorce from her husband Ashton Kutcher, amidst allegations that he cheated on her repeatedly, including on their wedding anniversary. People magazine reports that “sources” claim that Moore was willing to tolerate some infidelity, but that his flaunting of his affairs, in ways that made tabloid headlines, became inexcusable. While People doesn’t come right out and say they had an open relationship, that is the implication. Whether it’s true or not, the idea highlights a reality whether you’re famous or not: there exists an aspect of our sex lives and how we present them, that’s public, and an aspect that’s private. For some they may be one and the same, but for others, they are two very different but overlapping pieces of our erotic puzzles.

This item of gossip caught my eye, because it gets at the heart of the ways sex occupies both the public and private spheres, often straddling them, moving from one to the other at our whim. Some of us like to let it all hang out (raising my hand) and some of us prefer to keep people guessing, but even if you count yourself among the latter, your sex life is still fair game for speculation.

Especially when it comes to taboo topics we fear might get us in trouble, many people, even those who’ve publicly staked a claim to sexuality, are not comfortable being so “out there.” Three writers who’ve written books about sex have asked me to remove YouTube videos of them reading at my former reading series In The Flesh, because they feared repercussions at work or in their relationships. Potential erotica authors frequently query me concerned about privacy issues around pseudonyms, because were their erotica persona to become public, they too could lose jobs, friendships and respect. We may like to think we are living in an anything-goes era when it comes to sex, but we aren’t quite there yet. I wonder what our world would look like — and if couples like Moore and Kutcher would still be together — if we were less judgmental and more open to an array of sexual desires. Instead, topics like crossdressing, bisexuality, and BDSM still trigger shame in many people. I get the need for secrecy, especially if it’s about a topic that one fears (or is sure) will be misunderstood, all the more so with something like role-playing, which has the potential to reveal so much about who you are as a person, not just what you like to do.

It’s not just those with a secret life, who want to stay under wraps, who are affected by these double standards; recently, I found myself making assumptions about someone else’s sex life. Susie Bright posted an interview on her blog she conducted with sex educator and Opening Up author Tristan Taormino, and I was surprised to read about her own jealousy issues. “I am often too quick to say ‘YES’ to my partner’s request to do something with someone else. It’s like I want to facilitate his pleasure and happiness and not be jealous, so I jump right to ‘Sure, that’s great! Go for it!’ without actually thinking it through,” said Taormino. Of course it makes sense that even Taormino would have dealt with “some major doozies,” in her words, when it comes to jealousy, but I, myself, fell into the trap of thinking along the lines that since she’s an expert on the topic, she never faces the same trials and tribulations as the rest of us.

Anyone even marginally in the public eye, not just those of us who work in the field of sexuality, is going to be subject to some speculation, whether to your face or not, about what your sex life is like. That curiosity is natural, but it can create a gap between other people’s fantasies about our sex lives (or perhaps our own idealized visions of them) and reality. This begs the question of whether we’re secure enough in our own sexual desires to not care about what others might project onto us.

It makes sense that what we might tolerate within the confines of a relationship might not be something we can tolerate out in the open, where we are subject to judgment and prejudice. Sadly, I suspect many people still hear “open relationship” and process it as “nonstop orgy seven nights a week,” even though it may more likely mean a once-in-a-while hookup, or several dedicated relationships (or perhaps you are enjoying that nonstop orgy, and more power to you!).

The nuances of polyamory are still not quite accepted, even though we know that the divorce rate in the United States is extremely high; cheating seems more understood, at least as a necessary evil, than actually doing the work of negotiating some variation of nonmonogamy. Plus, if you’re in a heterosexual marriage, the default assumption is that you are monogamous; I know plenty of couples who from the outside look like the typical man and wife, and reap all the benefits of societal blessings as such, but practice nonmonogamy. I’m not criticizing that arrangement, but I would love to see more famous and non-famous folks acknowledge the challenges of relationships generally, open and otherwise. Yet when they do, they’re skewered with I-told-you-so scolds, like Chelsea Handler’s not-so-smart quip to Piers Morgan, “They probably had a lot of threesomes that led to twosomes without Demi and that leads to a divorce.” The notoriously sexually conservative tabloid Hollywood Life did their best to sound aghast about this utterly sensible quote by Angelina Jolie: “I doubt that fidelity is absolutely essential for a relationship…It’s worse to leave your partner and talk badly about him afterwards. Neither Brad nor I have ever claimed that living together means to be chained together. We make sure that we never restrict each other.”

On a personal note, because I’ve written extensively about my sex life, plenty of people I meet, whether we’re familiar with each other on an intimate basis or not, think they know exactly what I’m up to — or want to be up to. The fact is, even I don’t know until I’m in the moment. Sex occupies a special place in our society in that we all think about it, and we are all subject to the larger culture’s ideas of what sex is, and what it isn’t, what’s acceptable and what’s not. We may not agree with those terms, but they do shape our sexual educations (in school and in practice), our laws and our societal taboos. I don’t want to see a total collapse of public and private, because there is something special about the private moments and memories, the ways a particular erotic revelation can make you feel instantly closer to a lover. But I do want to see us open up the public space to talk about sex and acknowledge that it’s not a one-size fits all endeavor. Otherwise the schism between public and private will continue to challenge us, rather than us challenging it back.