Tuesday, March 5, 2019

Undoing the MDS Project



“While you are in this facility, how important is it to you to …?”
“Very important.  Somewhat important.  Not very important.  Not important at all.  Important but can’t do or have no choice.”

I’d like to turn that question around …

“As an activity professional, how important is it to you to use up to date interview techniques that directly generate meaningful information about your residents?”

  1. Very important (If I’m going to be held accountable for the numbers generated by the MDS, I want them to reflect reality.)
  2. Somewhat important (I’m kind of tired of residents telling me things that are the opposite of what their activity choices will be.)
  3. Not very important (Don’t know, don’t care.)
  4. Not important at all (I truly don’t mind wasting a lot of time to satisfy the folks at the Center for Medicare/Medicaid Services.)
  5. Important but can’t do or have no choice (The system is crippled, but nobody cares what I, a lowly AP, have to say about it.)

Back in the 1950s, a fledgling psychologist named Daniel Kahneman was given the job of evaluating the new soldiers in the Israeli army.  They had been using interviews, but the results were pretty meaningless.  They didn’t help predict how the soldiers would perform.  Sound familiar?  So, he designed a new type of interview.  Michael Lewis describes the situation in his book, The Undoing Project.

“He [Kahneman] told them to pose very specific questions, designed to determine not how a person thought of himself but how the person had actually behaved” (80).

Suddenly, they were generating more meaningful data, data that had predictive value.

In the 60s and onward, Amos Tversky joined Kahneman to study how people make decisions.  Again, Lewis describes their findings:

“When people make decisions, they are also making judgements about similarity, between some object in the real world and what they ideally want” (114).

Residents in long-term care are asked to make decisions about which activities to participate in.  At that point, they are making judgements about the activities offered compared to what they ideally might want.  But judgements about similarity, Lewis continues, are related to the features we choose to compare, like how noticeable they are, the context we see them in, and how our brains classify them.  For instance, a resident who likes music might chose to skip a group music event because “it’s for old people who have lost their minds.”  He knows that other seniors, ones with memory/health issues are likely to be there.  However, if his room is close enough to the event, he might hear the music and excitement, change his mind, and show up.  Or not.  Decision-making is highly subjective and can change at any moment.

Tversky and Kahneman also realized that people make decisions based on the potential loss or gain they associate with the decision.  But to use potential loss vs. potential gain you have to start at some “reference point” of how you value where you are right now.  The problem is that the reference point can also vary.  It is basically a “state of mind” (275).  I don’t know about you, but my state of mind changes all the time.  I have good days and bad days, good moments and bad moments.  I remember skipping a good friend’s wedding back in the days when I was single.  It was only at the last minute that the prospect of sitting through a lively Polish wedding reception with no date suddenly sounded like a more of a downer than I wished to risk.

In the 1980s, I was in graduate school studying sociolinguistics.  We were trying to find out which language people would tend to use, and for what reasons, in societies that were multilingual.  Some languages enjoy higher prestige.  Some languages provide more credibility on the street. “Do not base your data on self-reported language preferences,” my professors told me.  Instead, we read about ingenious work-arounds that linguists use to ferret out when, where and why people choose to use one language instead of another.  I have interviewed people who claimed to always use a certain higher status language.  Only when pressed did they admit to using a lower status language for some situations.  Self-reported language use is more about self-image, than fact.  I suspect that the self-reported importance of religious participation also often falls into this category.  It’s a normative, a “what should be”, rather than what is.

Even major league baseball and basketball have learned to upgrade the metrics they study.  These teams compete during games, but they also compete to sign the best players.  Lewis documented how the 2002 Oakland Athletics baseball team competed successfully against teams with much more money to hire new talent.  They found that the normal statistics that scouts had been using failed to predict future performance.  The Oakland A’s knew what to look for and could get those players more cheaply because the big teams were overlooking them.  I wonder what metrics we could come up with that would more accurately predict future activity participation. And what if what is important to US, and what WE prefer, is more relevant to their participation than any data we could get from the residents themselves?

Do our residents maximize their utility?  Short answer – no.  “Maximizing utility” means that people will reliably attempt to get the most value for their expenditure, the most bang for their buck.  In the case of activity programming, it could mean the most happiness for the expenditure of time and energy.  Kahneman saw that economists lagged behind the psychologists because the economists

“…assumed that you could simply measure what people wanted from what they chose.  But what if what you want changes with the context in which the options are offered to you?” (278)

Just because someone used to enjoy doing something independently doesn’t guarantee they will want to do it with our “facilitation,” or in the nursing home environment at all.  Ever.

Lewis summarizes Kahneman’s conclusion about maximizing utility.

“…people’s anticipation of happiness differed from the happiness they experienced, and … both differed from the happiness they remembered.
… If happiness was so malleable, it made a mockery of economic models that were premised on the idea that people maximized their utility.  What, exactly, was to be maximized?” (351).

In 2001, Kahneman, the psychologist, won the Nobel Prize in …economics!

Scientists, psychologists, linguists, statisticians, sports analysts, and economists, have found that people in general are systematically irrational in their thinking processes and choices.  We don’t really know why we do what we do.  Making irrational choices is not an aberration.  It is normal.

Fast forward to 2018, and CMS still has us ask our residents to self-report on their activity preferences in the absence of any observable behaviors.  That is the MDS 3.0.  Then we are expected to relate that to the activity decisions the residents will actually make.  That is the activity care plan.  Then we are held accountable for any pattern of discrepancies between the MDS 3.0 and the residents’ actual behavior.  That generates a CAA and potentially a flag.

I know it’s possible to make lemonade out of a lemon and use the MDS interview to also generate a certain amount of activity history.  But that doesn’t make up for the amount of time and energy that CMS wants us to waste on the quest for that precious #1 – 5 that we have to try to coax from the residents.  There are better ways to get an activity history.  There are better ways to develop care plans.  And there are better ways to help activity professionals and residents develop good activity programs.  My activity director gave me a chuckle and a quote for this paper when she wrote to me that, “I am blessed to have you on our ALF actively engaging those folks into things THEY never knew THEY wanted!”

What am I saying?  Activity professionals are made to use methodology that is as much as 60 years out of date.  We are held to bad science that academics who study human behavior have spent whole careers discrediting.  There is a big difference in why our residents do things as opposed to why they think they do them.  Study after study has shown that nobody is a rational decision-maker.  So, who at CMS still thinks that you generate meaningful numbers by asking our residents “how important” something is?  And how long are we going to go along with it?

©Donna Stuart, ADC        January 21, 2018, 2019

Lewis, Michael. The Undoing Project. New York: W. W. Norton & Company, Inc., 2017.

https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/NursingHomeQualityInits/Downloads/Archive-Draft-of-the-MDS-30-Nursing-Home-Comprehensive-NC-Version-1140.pdf

No comments:

Post a Comment