Successful Self-Experimentation

Diet and lifestyle changes are ideally made with the guidance of an experienced and trusted practitioner, but sometimes that’s impractical or impossible. Sometimes, you don’t have a practitioner like that near you, or you haven’t found one who suits you yet. Sometimes, you hear so many good things about a particular diet, exercise plan, or herbal remedy, you just have to see if it lives up to the hype. So you decide to give it a try for a while and see how it affects you—you set out to conduct an experiment on yourself.

Introduction

It sounds simple enough, but there are a number of pitfalls with self-experimentation that can cause you to come up with confusing, incomplete, or unjustified conclusions. As a result, you might continue with a particular behavior that isn’t really helping, or you might dismiss as ineffective something that was actually beneficial—in both cases, defeating the purpose of your experiment.

How can this be prevented? First, we need to recognize the potential mistakes. Then we can find tools and strategies to help us avoid them.

self-experiment or self-justification?

There are some valid concerns about using self-experimentation as a technique for sorting out healthy and unhealthy diets and habits. Foremost among them is the prodigious human capacity for self-delusion and susceptibility to various forms of cognitive bias.

Andrew of Evolvify has written about what he calls “the self-justification diet”: “Bias makes it hard enough to train scientists to draw useful insight from experiments on others. Turning everyone into objective simultaneous experimenters and experimentees strikes me as an infinitely utopian endeavor.”

Chris Masterjohn comments in the same post: “A proper self-experiment, as opposed to a flimsily interpreted experience, would be to put oneself (preferably blinded with the help of someone else if possible) through several treatment and control trials in a randomized order, in which the “n” becomes the number of trials rather than the number of individuals. Then one can use statistics to determine if there is a treatment effect.”

Kurt Harris suggests that evidence from self-experimentation should indeed be the “ultimate” consideration—in the sense of the last thing considered, arguing that subjective experiences are too unreliable to be trusted and that scientific evidence should be the basis of all dietary and lifestyle changes. He goes on to say that “the scientific reasoning should be sound and there should be evidence. And then N=1 should be applied with caution, with emphasis on objective measures of health.”

does n=1?

If you followed any of the links in the last section, you saw the phrase n=1 come up a lot. In the current vernacular of the “paleosphere” (bloggers writing about various evolutionary-history-inspired diets, where this topic comes up often), n=1 is the shorthand used for self-experimentation. The term comes from the notation used in scientific studies (specifically, cross-sectional studies, or studies that take a single set of measurements from a number of participants) to indicate how many participants were involved: a study with 42 participants would be n=42. So, any self-experimentation would naturally be an n=1 experiment.

As you may have noticed, n=1 often comes with a pejorative undertone in these discussions. The criticism is that such studies can’t be used to generate reliable conclusions, because they don’t gather enough data to make generalizations. No reputable journal would publish a study that was only n=1 — “that’s not data, it’s an anecdote.” But this criticism is a red herring.

Ned Kock of Health Correlator wrote a nice article not too long ago, explaining why calling self-experimentation n=1 is inaccurate. In reality, he explains, n equals however many data points you give yourself; you are in fact doing a longitudinal study (a study that takes several measurements from participants over a period of time) with one participant. Just like with cross-sectional studies, greater n values are more useful. So if your “n=1” really is an n=1 — if you really only use a single experience (data point) as the basis for your conclusions, then yes, you’ve only reached the level of anecdote and you’re not justified in making claims. But if your “n=1” is really an n=10 or n=100, you’re that much more justified. (Refer back to Chris Masterjohn’s comment quoted above, where he says that “n” becomes the number of trials rather than the number of individuals.)

Why does this settle the criticism? Because it changes the terms of what we call n=1 and what we expect of our self-experimentation. We might still use the term n=1, but only in its colloquial sense; self-experimentation is the more exact term, and we can set down some parameters for what does and does not count as “real” self-experimentation.

subjectivity, objectivity, and practicality

An immediate reaction to first feelings (e.g. “but I feel good when I eat pizza!”) doesn’t constitute an experiment—you haven’t done the work. Delayed-onset and invisible effects haven’t been accounted for: are you feeling an endorphin response due to your undiagnosed gluten allergy? Is your immune system attacking your own tissues because they bear some structural similarity to food proteins that are passing undigested through your damaged intestinal walls? Will you have a migraine tonight, or be constipated tomorrow? And if you are, will you remember that the last three times you ate pizza, you had the same symptoms?

It would be nice to know for sure. It would be nice if we could have objective, clear-cut answers to every dietary and lifestyle question we can come up with. But objectivity like that isn’t cheap (and so, for many, it isn’t accessible), and in many cases it isn’t even possible.

There are some constants, and some concretes: trans fats are bad, green leafy vegetables are good. But more to the point, there are some areas where a large range of individual variation comes into play. Those are precisely the areas where self-experimentation is most helpful and indeed needful—and where, conversely, reliance on (sparse, contradictory, disputed) evidence from studies and one-size-fits-all prescriptions are least helpful.

How many grams of carbs per day makes a diet “low carb”? Can raw dairy be part of a “paleo” diet? Should nightshades be avoided entirely, or can they be eaten regularly? Is white rice a “safer starch” than sweet potatoes? There is no consensus among researchers or enthusiasts about these questions and more like them. Many have never been studied at all—either they were never considered, or they were disregarded as obvious due to the assumptions of conventional wisdom. And in the cases where they have been, the variation between individuals is high.

In herbal medicine, there are similar questions that can be most easily answered (for a given individual) by experimentation. Will valerian help me sleep or keep me up all night? Will the licorice in this tea blend exacerbate my high blood pressure? Is skullcap or blue vervain better for my particular breed of anxiety? An herbalist will be able to see the factors that make one or the other answer most probable, but the interaction between person and plant is complex and often surprising. If you’re on your own and want to get started right away, you’ll just have to try it and see—and that’s no problem! With a little work, you can make sure your self-experiments are successful.

perception, reflection, and connection

If we boil the many criticisms in the anti-n=1 posts and their attendant comments down to their fundamental ideas, we get the following core argument:

• self-experimentation is reliant on subjective experiences
• subjective experiences are susceptible to cognitive bias
• cognitive bias leads to unchecked and incorrect conclusions
• ∴ self-experimentation leads to incorrect conclusions

In logical analysis of an argument, you can either dispute the premises themselves or show that the conclusion doesn’t follow from the premises. In this case, the argument is of simple and direct form (if A then B, if B then C, if C then D; ∴ if A then D), so we must concentrate on the premises. Three premises means three points of vulnerability:

• reduce reliance on subjective experience
• reduce susceptibility to cognitive bias
• justify conclusions more thoroughly

Which is to say: we must work to develop skills of perception, for subjectivity shades into objectivity with sufficient perceptive depth and breadth; reflection, for cognitive bias can [only] be overcome by cognitive discipline; and connection, for discerning a distinct pattern gives us greater certainty in our conclusions.

Perception

The beginning is the development of skillful perception.

When engaging in self-experimentation, there are certain skills that must be put into play in order to get useful results. First are the sensory skills that form the borders of our interactions with the external world, and can give us insights into our own internal world as well.

developing sensory skills

We consider a measurement or assessment with criteria for reliability: if it can be tested, if it can be communicated, if it can be reproduced, we call it objective. But what, exactly, is being tested, communicated, reproduced? A position on a gradient, perhaps, or a density of one type of cell in a sample, or a ratio of one biomolecule’s prevalence to another’s; a set of perceptions. Quantified, objectified, given specificity and symbolic value so as to be compared to benchmarks—but perceptions nonetheless.

Measurement is a perceptive act. Microscopes and stethoscopes, antigen assays and whole blood counts, all are tools to enhance, extend, or expand our perceptions. When we achieve a threshold of perspective-giving breadth or focus-finding depth in perception, we refer to it as a measurement. We should not forget that the tools we use do not change the fundamental nature of the act.

Those mentioned above are external tools; there are internal tools as well. Meditation traditions have taught for ages that perception is something which must be practiced and developed in order to be reliable. With such tools, we can take internal measurements as well as external; with such practice, we can approach, and perhaps cross, the threshold from subjectivity to objectivity.

To bring that down to [h]earth, if we’re willing to engage in a disciplined effort to become more perceptive of what is going on in our bodies, we can learn to detect its responses to our actions, and use them as guides when making changes.

mindfulness and intuition

One internal perceptive tool is the idea of mindful eating. A shorthand for this tactic is eat when hungry (usually paired with sleep when tired). Simple as that sounds, there is a little more to the story.

When practicing mindful eating, one goal is to always pause for a moment before eating anything, to ask yourself, “am I actually hungry right now?” and “is this what I want to be eating?”, to consider a moment, and only then to decide whether to eat it or not. This can be very helpful when trying to curb boredom-snacking between meals and other unconscious eating habits.

What has become automatic must be reconsidered and reassessed. Over time, the need for conscious intervention in the assessment diminishes. You find you have to ask yourself the question less often, and spend less time deciding how to answer it, until it passes into [p]reaction. Mindfulness in a particular area of life has a habit of developing into intuition about that area.

As it turns out, intuition is a skill—something which is understood intellectually and initially learned by focused force of will alone, but which operates more efficiently and effectively if it is practiced to the point of becoming instinctual and reactive. Martial arts and symbolic logic are skills. Herbal formulation and constitutional assessment are skills.

Intuition is a skill of (semi|un)conscious perception and pattern recognition. Others, less liminal in their nature, come into play during self-experimentation as well.

interoception

Developing and exercising all the senses is important. Students of herbal medicine learn to identify plants and their characteristics organoleptically—using their senses as a fine-tuned chemistry set. The bright smells of warming aromatic oils, the sweet-tasting slipperiness of mucilaginous polysaccharides, and other sensory input give us information about the plants and what they can do in the body. But we as self-experimenters can also develop the direct sensory experience of our internal condition. Interoception or visceroception is a fancy (or, if you prefer, precise) name for this.

I found this book while looking for the current term for this sense, and I’m looking forward to reading it. From the foreword: “Mood and emotion, according to Ádám, can be influenced by bodily feedback outside of awareness. Visceral activity—which can be the result of digestion, exercise, and a variety of known and unknown factors — can thus specifically effect mood states.”

This seems fairly well accepted now, as evidenced by articles like this one from Current Opinion in Neurobiology in August 2003: “In humans, a meta-representation of the primary interoceptive activity is engendered in the right anterior insula, which seems to provide the basis for the subjective image of the material self as a feeling (sentient) entity, that is, emotional awareness.”

The fact that this area of the sensorium is generally considered “outside of awareness” doesn’t necessarily mean that it is irrevocably so. This may be so in the majority of people (and/or the majority of the time), but in the same way that it is possible to retrain atrophied muscles in the foot and learn to lift each toe individually at will, it should also be possible to [re]develop sensory sensitivity to the internal body—both visceral and emotional.

Also from the foreword to the Ádám book: “One of the problems of symptom reports may well be that they are, by definition, conscious verbal labels. The mere process of tagging visceral cues to language may distort the visceral information.” Here, too, we should recognize the possibility that we can learn to give name or number to our sensations in a reliable way.

Learning to do so requires practice; that practice is embodied in reflection.

Reflection

Reflection is a bending back. It requires some flexibility.

Reflecting is intimately bound up with recording. If a recording does not reflect well its original, we say it has low fidelity. We prefer the highest degree of fidelity possible, as this gives us the greatest amount of useful information.

Reflection is similarly bound to recollection. Recollection is a reconstitution, and this always involves some adhesive or binding agent, which is an addition to the original component parts and which may keep them from fitting together as closely, or covering the same range of motion, as they once did.

There is an inertia to recollection in that it tends to emphasize only certain recurring surface patterns and diminish those that are less common or more complex. A good record is information-rich enough that its matter-of-factness resists this inertia and keeps the salient complexity of the original experience close at hand.

All that is to say that the quality of your record-keeping can make or break your efforts at self-experimentation.

for the record

Food journal, diet diary, intake-output log—these are all different names for the same indispensable tool. “Food journal” and “diet diary” are a little misleading, in that they give the impression that the only things to be recorded are what you eat. “Intake-output log” is more accurate but also more unwieldy. For purposes of these articles, I’ll just call it a record.

A good record includes all your food, drink, medications, supplements, bathroom visits, menses, sleep, exercise, stressors, emotions, and anything else that might be relevant.

It’s best to start this record before you make any changes, to establish a baseline.

What to put in your record:

1. Everything that goes in. Be detailed: include salad dressings, marinades, etc. Be as specific as practical, but it’s better to be inspecific than incomplete (i.e., “with oil” is fine if you don’t know what kind of oil). Include drinks, snacks, candy, herbal or vitamin supplements, etc. Be honest or it’s not useful! Make a note of the time.
   

   5:30pm – hamburger with mayo, ketchup, mustard, pickles, cheese. french-fries. coke.
   6pm – home made pasta and meatballs, salad, ranch dressing, bread with butter. red wine.
   7:30pm – calamus tincture, grassfed steak with onions and shiitake in olive oil, avocado with unrefined sea salt, olives, bacon-wrapped-almond-stuffed-dates, kale, beets. pinot noir.

2. Everything that comes out. It’s good to be descriptive if possible. Again, note the time.
   

   7am – urine: dark yellow, scant.
   10am – stool: loose, pale.

3. Everything you feel. Include everything—physical state (headache, stomach ache, PMS, tennis elbow) and emotional/psychological state (overwhelmed, tired, cranky, anxious, energetic). Note the time.
   

   10am – hungry, no time to eat.
   1:15pm – jittery, anxious.
   3:00pm – slumped, need caffeine!
   8:00pm – fight with roommate.

4. When and how much you sleep. Feel free to describe the quality of the sleep, whether you woke up in the night, whether you had dreams and what they were about, etc.
5. Any other details about the day, with the time if relevant. This might be classes, extra work, time at the gym, a note that your mom visited, your period started, you won the lottery or got in a car accident, . . . anything that affects you physically or emotionally.

symptom scores

For any chronic or recurring pains or ailments you have, you’ll want to include symptom scores in your record. We have a startling capacity to forget the extent of our pains once they are past us. Keeping a record of the severity we assign to a symptom helps us recognize when a therapeutic change is having effect.

This includes taking an inventory before you begin to make changes, noting any complaints you hope to address (e.g. eczema, brain fog, frequent headaches) and how painful or intrusive they are. Then you regularly reassess those complaints as your experiment goes on.

Your symptom scoring can be a simple 1-10 scale of least troublesome to most severe, or it could mean that you put your assessments into key phrases like “so bad I couldn’t sleep/work” or “recurs every ten minutes”. If your experimental changes are having some effect, you’ll see a shift in these values or descriptions over time.

This is the time to put your perceptive skills to work. Whenever you write something in your record, give yourself the moment to turn your awareness inward and take careful stock of what is going on in your body, and to reflect on how its present condition compares to how you’ve felt in the past, or how you feel usually. With a good-quality record that includes detailed notes on your intake and output, and meaningful descriptions of your felt experience, you’ll have all the material you need to make connections between them.

Connection

Humans are very good at pattern recognition. We’re so good at it, in fact, that we “trick ourselves” into seeing familiar patterns in clouds or in Rorschach blots. But we can’t recognize a pattern anywhere if we don’t look for one. Keeping the record isn’t going to do you any good if you don’t go back and analyze it.

Conversely, we might feel a sense of accomplishment if we make some change “for our health”, but we can’t have real confidence in it unless we can point to a pattern of cause and effect linked to it. Put those pattern recognition skills to work, and quell any doubts about the value of your adopted habits—whether those doubts are your own, or those of others who look askance at your strange new ideas about food, medicine, and movement.

baseline patterns

If you started your record before you made changes, you can look in it for baseline patterns in your health:

• Review your record, and look for anytime you felt particularly bad, particularly good, or particularly weird: migraine, nausea, panic attack, carpal tunnel; made a creative breakthrough, smiled at strangers, felt lucky; got turned around in my own neighborhood, forgot my co-worker’s name, lost an hour wandering in the internet. These could be isolated events, or “flare-ups” of chronic or recurring conditions you’re already aware of.
• Search for other instances of that same bad, good, or weird feeling in your record.
• Look at the days prior to those unusual feelings, to see if you can find any intake/output differences from your norm: changed my workout routine, tried a new restaurant, had ice cream for the first time in a month.

Some patterns will have a short time frame, like a few hours or a day. Had cheap take-out for lunch, headache at 2pm. Skipped breakfast Monday, snacked three times that day. Went to bed early on Wednesday, got a lot done on Thursday! Others will recur weekly. Pasta night on Thursdays, diarrhea on Fridays. Four hours’ sitting in the Tuesday status meeting, backache every Tuesday night. Still others could be on the scale of a month or longer. Drank Nettle and Friends every day this month, PMS & cramps were much less intense. But it should be possible in almost all cases to find some connection worth investigating.

(If you jumped right in and switched everything around on the day you started keeping your record, don’t worry. You can still get useful information to drive good results. But do take some time to reflect on your pre-change condition, and write down some notes about recurrent health problems, along with anything you suspect makes them better or worse.)

making changes

Generally, when starting a self-experiment, you’ll already have some information to work with. You’re not likely to spontaneously decide to start eating or exercising differently out of the blue: change takes work, and you’ll only put in the work if you have some expectation of benefit based on information you trust. Depending on your personality, this could be anything from a recommendation from a friend, to an article you happened across while browsing, to an intensive review of studies and reports on the subject.

So you’ll probably have some idea, when you start changing your diet and habits, about which areas of your health you’re hoping to affect. But, I would suggest that it’s best to keep your goals general. Don’t get too caught up in the adjustment of one number or another, be it your body mass index, cholesterol count, or one-rep max for the benchpress. Try to see the whole picture.

If you don’t have a specific health target in mind, you don’t have to worry too much about isolating variables—determining the individual effect of each dietary or lifestyle change. You’d do this if your goal was to determine whether a reduction of your sugar intake by 100g/day resulted in more favorable A1C numbers after 60 days, or whether taking 10ml of hawthorn extract three times a day reduced your incidence of heart palpitations by 90%—but that’s not your goal. Your goal is to feel better, and fast. Making multiple simultaneous changes will be more effective. You can always rechallenge some individual pieces later, if you have a nagging doubt that some aspect of your new program isn’t doing you all that much good.

Mark down in your record when you decide to make a change, and don’t be shy about telling your friends and family. Having it on paper, and letting others know about it, will keep you accountable!

making connections

After you’ve made your changes (gone soy-free, made your sleeping space as dark as possible, started walking 30 minutes a day) and—and this is key—stuck with them long enough for your body to adjust, then you can begin to evaluate their effects. The process is much the same as when looking for baseline patterns.

Review your record over the whole period of your experiment, and try again to find any bad/good/weird feelings. This time, keep a particular eye out for those symptoms you were hoping to address with your changes; note both their frequency and their symptom scores, and whether and how they changed over the course of your experiment. But look, too, for general trends: are you feeling good more often than not? Are your energy levels higher, overall?

It can be especially helpful to compare your condition now, after a month (or more) of your experiment, to your condition at baseline, before you changed anything at all.

Once you’ve found those trends and patterns, drawing a conclusion should be simple. If your habitual complaints have lessened, if a previously deteriorating condition has stabilized or begun to turn around, if you’ve gotten more done or felt less stressed — if you’re meeting your definition of health — then your experimental change has been working.

If it’s working, stick with it! If it isn’t, you can turn your attention to other options. If you’re not sure one way or the other, you have choices—you can try to identify and eliminate some confounding variables, stick with it for another period to see if effects will emerge after more time, or just leave it in the “undecided” category and focus on things that do have apparent success.

In any event, you’ve gained some useful information about how this change works in your own particular case, and because you’ve done your due diligence, you can feel confident in your decisions about it as you go forward.

Putting It Into Practice

In short: first perceive, then reflect, then connect.

In slightly-less-short: first, practice perceptive skills of intuition, interoception, and mindfulness; then, keep a record that accurately reflects your food, feelings, exercise, and experiences; then, identify health patterns at your baseline and after making changes, to see whether and where those changes are having effect.

In this article, I’ve described the process in a middle way: it could be done more formally, or less. You could measure and track your health parameters as closely as time, money, and technology allow—or you could rely more on general impressions, making note only of particularly exceptional feelings and events. It all depends on your personal inclinations and on the severity of the health issues you’re hoping to address.

practical practices

Here are a few pointers that can help make the process go more smoothly:

• Get a small notebook for your record, something you can always have with you. Keep it near you so you can always write down what you notice and do throughout your day. Tracking apps can also be helpful, and there are lots to choose from!
• Stick with your changes long enough to know if they’re working (unless you find yourself experiencing adverse effects). Giving up after too short a period will give you false negatives, the mistaken impression that something wasn’t working when it actually was. The body needs time to readjust to the “new normal”. In general, when eliminating a potential food allergen, you need at least a month, preferably two. When starting an exercise regimen, two weeks. Herbal medicines vary widely in their time to effect, but one month should be enough in almost all cases.
• Once you get a good result, don’t drop your recording habit. Keep it going for another period, even if you don’t change anything further. Particularly in the case of herbal medicines, you may find that constitutional effects build up over time, and the herb that was at first very helpful with one problem is starting to create another, e.g. a warming, drying herb can be helpful to bring balance to someone who is constitutionally cold and damp, but if taken continually it could begin to heat them up or dry them out. Watch that you don’t overshoot the mark!
• When eliminating possible food allergens, and getting inconclusive results—or struggling with psychological resistance and doubt—try a rechallenge. After you’ve strictly eliminated the food allergen from your diet for a full month or two, try eating it again and observe your body’s reaction carefully. You may find symptoms start to re-emerge which you didn’t have while you avoided the allergen.
• If you find something that really works for you, remember that self-experimentation means your conclusions go no further than your own experience. Don’t give in to the temptation to generalize. Be open to new information. If you’re solid in your own experience, you don’t need to feel challenged when others react differently.

for clinicians

For clinicians, the problems of insufficiently disciplined experimentation cut deeper. In this case, it’s you who has to keep track of things for the client—you’re the experimenter. (Ideally, you’re working to track these things together, but you should look on it as your responsibility to the client to make sure you do your part thoroughly.)

Make sure these tools are part of your client tracking process. Teach perceptive skills and strategies. Try to get clients to keep records when making dietary or other lifestyle changes, and spend the time to teach how to use them effectively. Get symptom scores on intake and all follow-ups, so you have some clear evidence to show that progress is being made, and help the client see the patterns in their health at baseline and after making changes. This will help you identify when your recommendations are having good results, when they’re making little difference, and perhaps most importantly, when they’re having undesirable side effects.

Self-Experimentation Works, IF…

Self-experimentation is a valid and valuable tool—but only if you do the perceptive, reflective, and connective work required. Mindfulness and intuition, sensory perception, record-keeping, and pattern recognition are essential skills for making your “n=1” experiments successful.