In this episode, part of our mini-series of sex during a global plague, we take a skeptical look at a paper published in the journal Leisure Sciences called “Less Sex, but More Sexual Diversity: Changes in Sexual Behavior during the COVID-19 Coronavirus Pandemic.”
Along the way, we talk about the null hypothesis, the scientific method, and evaluating a research paper.
Franklin: Hello, and welcome to Skeptical Perverts, the podcast where we look at human sexuality through an evidence-based, skeptical lens! I’m one of your hosts and part-time mad scientist, Franklin Veaux.
Shara: Hi! I’m your co-host and Renaissance cat, Joreth! I have a background in human sexuality and relationship communication, I’m kinky, solo polyamorous, on the ace spectrum, chicana, feminist, my gender identity is “tomboy”, and my pronouns are she/her but I use masculine titles.
Eunice: And I’m Eunice, your friendly neighbourhood queer, kinky, solopoly, demisexual, East Asian Brit, ready with my pot of tea and a healthy dose of genteel snark!
Franklin: Today we’re returning to a topic that’s seriously affected all of us: sex in the time of an international plague. We, your hosts, are all part of a globe-spanning extended romantic network, so when this globe-spanning pandemic got going, it hit pretty close to home.
Eunice: Oh, and this is part 3 of our Sex in the Time of Pandemic mini series: Sexual Diversity! If you haven’t listened to parts 1 and 2, you’ll probably want to go back and check them out too, but you don’t need to listen to them in order.
Franklin: Today we’re looking at an interesting study, Less Sex, but More Sexual Diversity: Changes in Sexual Behavior during the COVID-19 Coronavirus Pandemic. The researchers were looking into the hypothesis that the pandemic may be responsible for a surge in people experimenting with new kinds of sex, especially folks without a live-in partner or people in long-distance relationships.
Given how hard it is to get research money for anything related to sex, the fact that this research exists is a bit surprising. I guess people have to die before funding frees up for sex research.
Eunice: Not sure that’s entirely true. 1980s, anyone? That’s the last time we had a global pandemic that affected people’s sexual behaviour, anyway. And research on that took a long time to get funded.
Franklin: Ouch. Okay, fair point.
Eunice: It’s definitely true that this study came out remarkably fast though. So what does the study actually say?
Franklin: Before we talk about the study’s conclusions, I think we need to take a look at that hypothesis, the demographics of the survey participants who volunteered for the study, and how they used the data they collected. And be aware, this podcast might be all over the place, because the study we’ll be talking about is a bit all over the place.
Eunice: Well that’s an understatement and a half. It’s so understated it’s almost British.
Shara: This study is so unfocused that it couldn’t find its ass with both hands and a map!
Eunice: Isn’t that Franklin? Oh wait, I think that might be me too. Hang on, are you the only one of us who has any sense of direction??
Franklin: I know precisely where my ass is, thank you very much. It’s everything else I can’t find. I still get lost in my own neighborhood.
Shara: Hah, yeah, I think I am the only one here with any sense of direction! OK, so this study is so unfocused, my old director just called me from across the country to yell at me about checking the focus on the screen. (I may have been watching too much Mrs. Maisel lately. Thank you and goodnight!)
Franklin: Anyway, back to the study! It’s called “Less Sex, but More Sexual Diversity: Changes in Sexual Behavior during the COVID-19 Coronavirus Pandemic,” which might make you think it was a study about the differences in sexual behavior during the pandemic and before the pandemic. The first issue I have reading the study is they talk a lot about the way people are having sex during the pandemic, and then talk about how many people changed their sexual behavior during the pandemic, but they don’t talk about how those changes compare pre- and post-pandemic. In other words, they’ll say thus and such many people tried something new during the pandemic, but how does that compare to how many people tried something new before the pandemic? Did the rate of trying new things increase, decrease, or stay the same? We don’t know; the researchers didn’t gather those data.
Eunice: I don’t know how they intended to answer any of those questions without getting baselines. Plus, getting those data is really important, but also really important is who you ask for that data!
Shara: Yeah I think another major point here is that the demographics are not at all representative of North American or world averages, in so many different ways! Like, in contrary directions, they manage to not be representative.
First of all, 71.1% of the 1,559 respondents were “female”. 71 PERCENT! While 23.4% were “male” and 4.5% were “nonbinary”. Next we have 84.1% of respondents being white with the remaining 16% being a mix of other races. Neither of which matches the general US population which tend around 50% each. Meaning that the population is assigned “male” and “female” at very close to 50/50, and while white people continue to outnumber each other individual racial demographic, added all together, white people are very quickly approaching the 50% mark, with projections estimating that they will become an ethnic minority in the near future (assuming you add all the other ethnicities together). So these demographics don’t match the general population.
In addition, the income stats are WAY not representative. The largest majority (and not by a huge margin) at 24.8% is people with a median income of $20-55,000, which is pretty solidly middle class, and I’m under the impression that this income group is steadily declining.
Eunice: Oh yeah, and the next biggest group, at 24.6%, is $100,001 to a quarter of a million dollars. That’s wild. The two groups in the middle contain less than 15% of participants each! How does that gap even happen?! I’d also like to point out here that your three fabulous hosts are really not people who have any idea what $100,001 to quarter of a million dollars annual incomes would even look like, so, you know. Anyone wanna change that, get in touch!
Shara: I would love to test the hypothesis that there is a cap to the amount of money that brings happiness, just sayin’.
Franklin: The study says participants were recruited via Internet-based snowball sampling. Snowball sampling is a technique where you get participants to recruit other participants, like a multilevel marketing scheme for science. The problem is demographic clustering; people tend to socialize with other people who are like themselves. So if you seed your study with a particular kind of participant and rely on snowball sampling, that’s the kind of participant you’ll continue to get.
Shara: So that basically skews the demographics in a pretty non-liberal way – white, middle class to wealthy, women. But then we look at orientation stats, and we see only 52.7% heterosexual with everyone else fitting under the queer umbrella. Which skews the data in a decidedly *liberal* direction, seeing as how the US population is somewhere around 95-ish% straight.
Franklin: Looking at these demographics, it really seems like they reached two demographic clusters: Middle American suburbs, and urban San Francisco. That explains the weird peaks in the income distribution and the rather strange distribution of respondents who self-identify as straight vs. queer. It’s like they took these two groups and said “on average they’re a fair distribution,” which is a bit like putting one foot in hot water and one foot in ice water and saying “on average, I’m comfortable.”
Eunice: Mean, median, mode, folks—pay attention to which form of average you’re using!
Franklin: “My comfort has a bimodal distribution.” You know, sometimes when I’m around the two of you, my comfort DOES have a bimodal distribution.
Eunice: So on average, you love it? Anyway, one of the reasons we keep comparing to the US population is that the majority—73.4%—of respondents are from the US. For a survey that attempts to extrapolate to all Western populations, it doesn’t really take into account that from everything I’ve seen, the US is something of an outlier in its attitudes towards sex compared to the other Western countries.
Franklin: In general, I’m not convinced the study data really supports the study’s conclusions, demographic weirdness aside.
Eunice: Real talk here: I’m not entirely sure that they waited until the data came in before writing those conclusions. Oh wait, is that libelous? I probably shouldn’t say that, huh. But seriously, looking at the tables of data in this study…I mean, I know I’m bad at statistics, but I really don’t think I’m as bad at statistics as these tables are implying to my brain. My brain actively hurts trying to read these.
Shara: To be fair, I think everyone is bad at statistics, because our brains (generally speaking) are just not made for statistics, but holy fuck is this an unreadable set of tables! I mean, it shouldn’t have taken me as long as it did just to figure out WHAT THE FUCK ARE THEY REFERENCING IN THIS TABLE?!
Franklin: So let’s dive into some problems with this study. But first, let’s talk about p-values.
Shara: Because with this study, you really need to understand p-values to understand just how bad this study is.
Eunice: So what are p-values, exactly?
Franklin: When you read a study, you should see p-values listed next to conclusions. Put simply, and handwaving over some nuance, the p-value is the statistical chance that the null hypothesis is correct; that is, the chance that the data might just be random, and the data don’t support a connection between the things you’re asserting are connected.
High p-values mean the conclusion is not well supported, or to put it more properly, that the null hypothesis is better supported or can better account for the data you’re seeing. Low p-values mean it’s less likely the null hypothesis is correct. So if you say two things are correlated, and you say the p-value is .001, that means you’re saying there’s a 0.1% chance that your data are just random and the two things aren’t correlated. On the other hand, a p-value of 0.8 means about an 80% chance the null hypothesis is correct and these things aren’t really correlated.
Shara: So, is this what you’re saying? A study starts with a hypothesis, which is a statement of what the researchers are investigating. Then they set about trying to disprove that statement, and if they can’t disprove it, then the statement is likely to be correct. If something in the study has a p value of, say, .05, then it means that there is only a 5% chance that their statement is wrong, therefore they are 95% confident they’re onto something?
Franklin: Basically yes, and then we get to what the null hypothesis is.
Eunice: OK, so what is a null hypothesis?
Franklin: Generally speaking, the “null hypothesis” is usually “there is no relationship between these things” or “there is no difference between these two groups.” So if you say “men are better singers than women,” the null hypothesis is “there’s no meaningful difference in the singing skills of men and women.”
If your p-value is .05, that means there’s a 5% chance that your data might show what they show if the null hypothesis were true. If it’s .8, there’s an 80% chance your data might show what they show if the null hypothesis were true.
This is simplified, of course, so if you’re a statistician and you’re cringing right now, that’s why.
Shara: OK, so if your p-value is .8 and your hypothesis is that men are better singers than women, then that means there is an 80% chance that men are not, in fact, statistically better singers than women according to your data, because your hypothesis is that there IS a relationship between men and women – that men are better, while the null hypothesis states that there is NO difference between the two groups, right?
Eunice: That makes sense. And I know the usual p-value that’s typically used in a lot of studies is .05, meaning that it needs to be at least 95% likely to be true for the researcher to be confident about their conclusion. So what hypothesis are they actually trying to disprove in this study, and why are they using the p-values they chose? Because it feels like it should not have taken this much effort to work it out.
Franklin: So let’s circle back and see how that plays out in this study. In this particular study, the researchers started out examining whether men or women were more likely to make a new addition to their sex lives. The data have a p-value of .833 which means the null hypothesis, that there’s no difference between men and women adding new things to their sex life, is most strongly supported.
Shara: OK, so they started *out* looking at whether men or women were likely to make new additions (and we’ll get back to that point), but then they used the results of a survey with a whopping 71% self-identified women? Can you really compare, at that point, between those two genders when their representation is so imbalanced?
Eunice: So I know it sounds like we’re putting a lot of effort into ripping apart this study, and all this talk of stats and p-values and such is probably sounding a bit dry right about now, but this study is a really great example of what not to do, and we’d love to talk about what we’d prefer to see in sex research instead.
Shara: In order to understand why we want to see what we want to see, we feel that we need to dig in to why this study was so bad, so let’s get into that for a bit before we discuss what we’d rather see in sex research. In this study, they found something interesting. They found that, basically, the frequency or *amount* of sex people were having went down, but the *quality* of their sex life did not.
“Many participants (43.5%) reported a decline in the quality of their sex life, with the remainder reporting that it either stayed the same (42.8%) or improved (13.6%). Average frequency of solo and partnered sexual behaviors significantly decreased compared to past year frequencies. Thus, during this period of widespread restrictions on movement and social contact, frequency of sexual behavior—an activity often pursued for pleasure and leisure purposes—decreased on average.”
So if you look at those numbers, decline in quality and no change in quality is very nearly identical – 43.5% vs. 42.8%, but when you include in improvement at 13.6%, decent quality of sex was actually the majority, then they jump to talking about a decline in frequency as if it’s related.
Eunice: See, this is one of the reasons I found this study so hard to understand. It jumps from a conclusion about quality to talking about quantity in the very next sentence without actually explaining why. It just puts two different conclusions side by side, implying that they’re connected. In fact, it’s practically encouraging you to draw the unconscious link between the two!
Shara: Yeah, I feel like this quote here is guiding me to the conclusion that “pandemic sex sucks” by linking a decrease in frequency with nearly equal levels of quality in sex.
Franklin: One of the things that really worries me is the table labeled “Frequency of new additions to participants’ sex lives during the pandemic.” This table contains such extreme and far-out new activities as “Tried a new sexual position,” “Shared sexual fantasies with a partner,” and “Took a shower/bath with a partner.” If these are new activities people are exploring, I weep for humanity. And what’s even more frightening are the raw numbers. This is a study with thousands of respondents, and the number of people who say they tried a new sexual position? 49. Shared sexual fantasies? 41. I mean, not everyone writes science fiction novels based on shared sexual fantasies, but still.
Eunice: OK, but looking at those numbers, and bearing in mind this group probably skews more liberal given they answered an online survey about sex, it might just be that they’ve already tried this stuff before, so it wouldn’t count as ‘new’ behaviour. So yeah, maybe only 49 out of 1559 participants reported having tried a new sexual position during the pandemic, but they all presumably tried a new sexual position for the first time at some point in their sexual experiences, right? They didn’t exactly come out of the womb having tried erotic lotus with upside down cowgirl and one-eyed banana, surely? (FYI I totally just made that up so I have no idea what that position would look like, write in with your suggestions!)
Shara: Right, I mean, just looking at physics and biology, considering the amount of sex I’ve had over my lifetime, I have literally tried every sexual position that’s even possible for my body to get into, so “tries new sexual position” is probably never going to happen to me (for the first time) again. And as I age, sexual positions will only drop off my repertoire, not get added. Hell, at this age, with my hips and knees, just regular old cowgirl is challenging!
Franklin: There goes my plans for trying the Monkey with Lotus Blossom and Chainsaw with you.
Shara: Well, we still have my penchant for taboo places!
Eunice: The beach is on my taboo list, mainly because I never want sand in those places ever again.
Shara: Fair point.
Franklin: I have a story about that. The sand…yeah, not a pleasant memory. Back to the study’s demographics. Young people as a demographic are also more likely to try new things in any given year simply because they haven’t had as much experience generally. When you’ve only been having sex for a year or two, many things you try are new to you.
Which is not to say there’s not still plenty of new things to try even as we get more experienced. You could live for centuries, doing something different in bed every night without ever repeating the same thing twice, and still not have time to do it all. That said, I’ll wager most people in their 20s try new things more often than people in their 50s, COVID notwithstanding. Separating the “I tried new things” from “I tried new things because of COVID” is a significant challenge, and it doesn’t really look like the study’s authors did that.
Eunice: Good point. I know it’s probably hard to retroactively find a control group, but it’s like they didn’t even try.
Shara: The study’s summary also says:
“Those who lived with a partner reported higher rates of partnered behaviors than those living alone, such as trying new sexual positions and acting on sexual fantasies; by contrast, living alone was linked to higher rates of virtual and technology-based behaviors compared to those with a live-in partner, such as sexting and sending nude photos.”
Shara: Ya think?
Franklin: In other shocking news, water is wet.
Eunice: Is it, though? Water gets things wet, but is it actually wet itself?
Franklin: That’s a philosophical question, which might be outside the scope of this podcast.
Shara: Then there’s this quote:
“Several psychological variables were linked to new additions; however, some associations differed across gender. Bivariate correlates of new additions are presented in Table 5. For men and women, reporting more sexual desire in the past two weeks was linked to new additions. For men only, loneliness was associated with new additions; for women only, desire for sex with one’s partner in the past two weeks, stress, and loneliness during the pandemic were associated with new additions.”
Franklin: If trying new things is statistically correlated with loneliness and stress, and unhappiness with your sex life is correlated with loneliness and stress, then you can’t make any inferences about whether trying new things is or is not likely to increase your sexual satisfaction, because both these things are linked to loneliness and stress. Loneliness and stress is a confounding factor that prevents you from establishing a causative relationship between trying new things and sexual satisfaction.
There’s something else I’m really curious about too. The study says
“Approximately one in five participants (20.3%) reported making a new addition to their sex life since the pandemic began. Most (62.7%) reported making one new addition, with 18.4% making two, 7% making three, and 12% making four or more.”
I’m really curious, though, how many people in any given year would report making an addition like this to their sex life WITHOUT a pandemic. Is that more than usual? Less than usual? Statistically normal? How are “new additions” counted? Is trying two new positions a single addition or two additions? What about two new sex toys?
Eunice: I think there’s an entirely reasonable desire here to tell a ‘Just So’ story, saying that of course when you’re bored and isolated as a result of the pandemic, then sex is a thing you would do more of. And if you have all that extra time, then you’d of course try new things. It feels logical. But without the baseline, how can anyone even tell? It’s just a story, not a hypothesis, and there’s no way to falsify it. It’s not even an entirely good story — I could come up with just as logical an explanation for the opposite claim. Also, here’s another interesting quote I spotted:
“Generally, only partnered activities were linked to improvements, with few technology-based activities showing any association.”
Now, according to this, only adding new partnered sexual activities showed any statistical likelihood of increased sexual satisfaction. Additional novel solo activities didn’t. Have they considered that it might be not the additional sexual activities in themselves, but the act of exploring new activities with your partner? Did they control for people who started new, non-sex-based, hobbies together and whether they had increased life satisfaction?
Franklin: Relationship satisfaction is about more than just sex.
Shara: Yeah, something that I think a lot of people have a lot of trouble understanding, judging by the number of Quora questions I get asking why anyone would keep dating a person in particular if they can get sex elsewhere, is that it’s not the sex that’s the important part of a relationship, it’s all the other things and the sex is usually a barometer for all the other important things. So to relate it to the context of this study, what if it’s not the act of new sex stuff that made people express satisfaction with their relationship, but the act of new STUFF, period? Because what if the very nature of adding new things to a relationship is the result, not the cause, of a relationship between two people who are actively engaged in their relationship together?
Something this study doesn’t even question is the direction of the causal relationship. Like, what if the reason why those who did NOT try new things didn’t try it because they got stuck together with someone they never should have been spending that much time together with, and now that they’re stuck together, they don’t even want to have any sex, let alone new and creative sex, which requires a trusting, intimate, vulnerable setting to even bring up the subject.
Franklin: So the takeaway here is looking at relationship satisfaction through the lens of sex misses the fact that both relationship satisfaction and sexual satisfaction may correlate with other factors, like trust, vulnerability, basic compatibility, and openness.
I feel that way reading this whole study. I’m glad to see people are putting time and attention into sex research, and I think research on how the pandemic has affected sexual relationships is potentially valuable. I am not convinced this study is particularly illuminating, though.
Eunice: I’ll be honest, this study had me so turned around I’m questioning my reading abilities, at this point. I feel like I was constantly trying to pin down an extremely slippery eel. If it feels to our listeners like we were all over the place, imagine what it was like from this side of the mic! Because that confusion was an extremely accurate representation of what it felt like to try and parse this study.
Franklin: It worries me a bit that you know what it feels like to pin down slippery eels.
Eunice: I make no guarantees as to how slippery any eels I may or may not have encountered in my past were.
Franklin: Now I want sushi.
Shara: This study was so difficult to follow that, even rereading our own notes explaining this study had me rubbing my eyes and questioning my own ability to follow scientific studies. Like, I swear I had much less trouble following this shit in school!
Franklin: Back to the topic, the biggest issues I see with this study are wildly nonrepresentative demographics and a tendency to draw a straight line between hypothesis and conclusion without accounting for possible related or confounding factors.
Eunice: Talking about confounding factors, the study also states that:
“The fact that those without a live-in partner tried more new activities is not entirely surprising because these circumstances likely necessitated more creativity with respect to pursuing sex for leisure. This likely partially explains why sexual minorities, racial minorities, and younger adults had increased odds of making new additions: all of these groups had significantly elevated rates of living alone.”
Did the researchers also consider that they’re the same groups that are more likely to overlap with key pandemic workers? Did they not wonder if the additional stress, physical risk, financial insecurity, etc etc etc, might have had an impact on their sex life? Did they even try to account for how much of the increased likelihood of “more creativity” was a result of living alone, versus those other factors? “Likely partially explains” is doing a huge amount of heavy lifting in that sentence, is all I’m saying.
Franklin: All of these are hard problems in any sociological research, but they seem particularly profound here.
Shara: So, basically, what we’re all getting from this research paper is that it attempted to generalize sexual behaviours from wildly, improbably outlier statistical groups, it then had such a mass of scattered p-values that it feels like the researchers rolled a D20 die and randomly assigned percentage points based on very excited rolls, and THEN took all of that and made sweeping causal statements without doing even the basic Freshman 101 exercise of considering for confounding factors. Would you both say that’s an accurate assessment?
Franklin: I think the study isn’t doing a good job of supporting its conclusions. A better designed study would be one that makes an effort to reach a more representative population and does a better job of isolating changes in behavior pre and post pandemic. Those are big asks, and make the research far more difficult, but I think the results would be more useful.
Eunice: Can I also put in a request for significantly better laid out tables of data? And graphs? Any graphs at all, please?
Shara: Dear gourd, those tables! Please don’t make me try to read those tables again! My eyeballs hurt.
Eunice: You ever try to read your own handwritten notes that you wrote at 4am on no sleep with a significant amount of caffeine and sugar in your system, for a class that you just realised during the lecture was maybe a step or two beyond your comprehension? Cos that’s what reading this study felt like to me sometimes.
Eunice: Can we never break down a study by these people again? Please?
Shara: OK, so, if we were designing a study on the question of sex, perhaps how the pandemic affected people’s sex lives, what would we be looking for, at a minimum?
- Better demographics in the study participants
- A better baseline of pre-pandemic sexual behavior to compare to pandemic behavior
- Better control of possible confounding factors
- Clearly laid out analyses and summaries
- Examination of whether the factors you’re measuring are connected or not
- Less eagerness to assume casual relationships that might not be supported by the data
- More legible tables
- Graphs! Not necessary, but really really helpful
Franklin: So that’s what we’ve got for this episode! What do you think? Send ideas, comments, new things you’ve tried during a pandemic, or suggestions for future episodes to email@example.com. And if you know someone else who might enjoy this podcast, why not share the love, by giving us a review on iTunes or Stitcher or your podcatcher of choice. You can also visit www.skepticalpervert.com for show notes, links to the transcript, and the studies we’re drawing from. And don’t forget to become a patron of the show by joining our patreon, which is linked on the website. The Skeptical Pervert is copyrighted and produced by Franklin Veaux, Eunice Hung, and Joreth Innkeeper, edited by Joreth Innkeeper, and the website and show notes are maintained by Franklin Veaux.
Eunice: The skeptical pervert: putting the ‘why?’ into sexy
Franklin: Why don’t I have any sushi?
Eunice: Are you looking to put eel into your facehole, Franklin?
Franklin: On advice of counsel, I decline to answer that question.