Quantifying Your Subjective Beliefs

In what year do you think the first U.S. Social Security check was issued?

Two people are asked this question, neither knows the answer with certainty, so they separately are guided through a process to represent both their best guess and degree of uncertainty, resulting in the two probability distribution curves below, dating back to 1776. Which person did “better”?

That depends on how we define “better”. Clearly, the flat-liner expressed much less confidence, but perhaps the confidence exhibited by the narrow-bander is misplaced. Let’s look at how each responded and performed.

Neither knew the answer off-hand, so they were asked to pick two extreme values, the minimal and maximal plausible values, and their single best guess, the 50th-percentile, such that in each of their respective views the correct answer was equally likely to be above or below that estimate. Next, each was asked to express their degree of uncertainty by specifying a 25th-percentile and 75th-percentile that properly reflects their uncertainty regarding this subjective distribution of possible answers. Surely each would agree that there is 100% likelihood the true answer is greater than 1 A.D., greater than 2 A.D., and so on — eventually reaching amounts believed highly implausible but nonetheless possible. Continuing on until reaching an amount such each believed there is about a 25% chance the true value is less than it and a 75% chance it is larger. That is Q1, a personal first quartile. Continue on until reaching Q2, a number representing an equal 50% chance the true value is lower or higher, and finally to Q3, signifying the personal view there is a 75% chance the number is lower and 25% chance it is higher. Notice that the respondent should be equally confident the true number is inside the range between Q1 and Q3 as that it is outside that range. That middle 50% of data is a subjective interquartile range (IQR). Overconfidence would be exhibited by a general tendency for subjective estimates of the IQR to exclude the true value more frequently than it includes it, so that the true value is lodged somewhere below Q1 or above Q3. For each person, we now have a five-poiont summary, the parameters of a distribution.

Ms. Bias
She recalls with assuredness that the Social Security program was a creation of Franklin Delano Roosevelt during his presidency, which she asserts as having run from 1933 to 1945. She believes the program likely started after his first year in office but before the Pearl Harbor bombing, and was equally likely to have occurred inside or outside a two-year window. Her five-number summary is:

The probability density function (pdf) and its cumulative values (cdf):

Mr. Variance
He confesses to having no idea. Pressed to offer something, he realizes that the answer must fall between 1776 and 2020, since it is a program of the United States government. That’s a start. Pressed further, he suggests the program likely was initiated following a significant historical event, lists five candidate events, and deems each event equally likely to have resulted in the initiation of Social Security. He maps out his thoughts, and decides those five dates correspond to the five-number summary that reflect his subjective beliefs:

So, Mr. Variance believes the actual date is equally likely before or after 1918, with a 50% chance it happened between 1865 and 1945, and 25% probabilities each for the 1776-1865 or 1945-1965 intervals. Applying these parameters to the general lognormal distribution results in the following five-number summary, subjective probability density, and cumulative probability distribution. Note the initial estimates are slightly adjusted. That is because the five-number summary is a set of parameters, which are fed into a program to simulate distributions.

In fact, as part of his “New Deal”, it indeed was FDR who signed the Social Security Act of 1935. Although the first SSA office opened in 1936 and the first Social Security taxes were collected in 1937, the first check was not issued until January 31, 1940. The recipient of that first check was Ida May Fuller, a legal secretary who paid a total of $24.75 in Social Security taxes during 1937-38-39, then received that first check for $22.54 upon turning 65. In an ironic precursor of financial strains on the program, she lived to age 100, collecting $22,888.92 from SSA over 35 years.

Truly unbiased subjective curves would be centered at 1940, with equal belief the actual year would as likely be later than earlier than 1940. Based on their respective belief curves, for Mr. Variance 70.0% of the area under the simulated curve fell earlier than 1940, whereas 95.7% did so for Ms. Bias. Both are biased toward an earlier year, but Ms. Bias far more so.

Overconfidence is characterized by subjective confidence in judgements or beliefs that exceed the demonstrably objective accuracy of those judgements or beliefs. Psychology literature suggests it is far more common than being properly calibrated, and evidence of underconfidence is relatively sparse.

More Complex Example

Let’s work through a more complex example. You are asked to estimate the dollar value of all distributions of U.S. Social Security payments to beneficiaries during May, 2020. Assume you had responded Q1 = $1 billion, Q2 = $5 billion, and Q3 = $50 billion. I can conclude from this that your IQR does not include the true answer, but cannot from this conclude you are overconfident. After all, if you are properly calibrated, being inside/outside the IQR is a 50/50 proposition.

Given sufficient opportunity, time and access to resources, one could construct an informed estimate of the dollar value of all distributions of U.S. Social Security payments to beneficiaries during May, 2020. There are eleven types of recipients, and for each there is a record of the number of beneficiaries and the actual payout to each in May 2020. To multiply the number of beneficiaries times the average payout for each category of recipients requires 22 input values, and the sum of the 11 products equals a total payout by Social Security of about $90 billion ($89,939,234,989.04) during the month of May, 2020.

There is a compromise strategy, more elegant than the loosely-anchored initial estimates of Q1, Q2 and Q3, but far less computationally demanding than the explicit enumeration above. A “fast and frugal” compromise strategy is to provide subjective estimates for a small and manageable number of components that have the potential to reasonably substitute for a complete listing of categories, and that might more reasonably be estimated by an outsider than the required 22 input parameters above.

For example, one could device a four-parameter estimation process:
1. World population
2. U.S. population as percent of world
3. U.S. Social Security recipients as percent of population
4. Average monthly payment per Social Security recipient.
* Total Outflow of Social Security Benefits ~ Product of the four parameters.

If preferred, the problem can be simplified further, replacing the first two parameters by directly asking for the U.S. population, thereby reducing to a three-parameter estimation problem (but at a cost: loss of information):
1. U.S. population
2. U.S. Social Security recipients as percent of population
3. Average monthly payment per Social Security recipient.
* Total Outflow of Social Security Benefits ~ Product of three parameters.

Applying the latter three-parameter estimation process, you now are asked to provide Q1, Q2 and Q3 subjective estimates for each of the three parameters, as shown below. Taking the best guesses (Q2), your new estimate equals 15% of 320,000, times $4,000 per recipient, equalling $192 billion — quite a leap from your original estimate of $5 billion. Error rates for each component are as follow:

A single poor estimate greatly impaired your analysis, as your average payout per Social Security recipient is almost triple the actual value. With only three parameters to estimate, a single poor choice can greatly distort results. Nobody expects highly accurate estimates, but with more parameters to estimate, the opportunity for errors to offset and provide a decent final estimate increases greatly. For example, another person might complete the same three-parameter estimation process, with these results:

In her case, the two larger errors are in opposite directions, and essentially cancel their distortions. The three component estimates esch have much more error than the remarkably accurate final estimate, due to error cancellation.

Regarding the matter of assessment overconfidence, let’s return to your respective component quartile estimates and their implications. The table below displays your estimates (with actual correct answers in bold and to the far right):

We now also have four independent estimates of Q1 and Q3 for independent questions, and note that the true values are outside the respective IQRs in all four cases, with three above Q3 and one below Q1. This provides preliminary evidence of overconfidence, although it must be noted that two estimates are relatively close to the Q3 bound and the sample size of n=4 can provide no more than tentative support for overconfidence, as opposed to a collection of several such three-parameter estimation processes.

Finally, note from the table above that you also were asked to provide estimates of the far tails to your original estimate of the dollar value of all distributions of U.S. Social Security payments to beneficiaries during May, 2020. In particular, while your best guess of the value was $5 billion, you also estimate your 2%-tile at $1 million, and your 98%-tile at $50 billion. Even this range does not capture the true value, as your 98th-percentile is little more than halfway to the actual value. This provides another indication of overconfidence. Also, your Q1 and Q3 estimates for the three-parameters imply a means of inferring your 2%-tile and 98%-tile beliefs regarding the total payout value. Since you believe there is only a 25% chance that a given true parameter value is below its Q1 estimate, and there are three parameters, you must believe there is a 25% * 25% * 25% (= 0.25^3 = 0.02, or 2%) chance that each of the three parameter values are below your respective Q1 estimates. The same applies to the other tail, at the 98%-tile. The visual display below provides an intuitive view of the calculations. Each quartile of a given parameter has an equal number of values, represented here by a 4×4 box with 16 cells per quantile. As we combine two parameters, we are left with 25% of the original cells below Q1 and above Q3, or four cells. Combining three parameters reduces cells by 25% again, from four down to one (1/16th of 16; 1/64th of the original 64 cells).

Applying this logic to your estimates, the implied percentiles are shown below. Your Q1, Q2 and Q3 estimates for each of the three parameter questions imply that your best guess estimate of the dollar value of all distributions of U.S. Social Security payments to beneficiaries during May, 2020 is $192 billion — quite the contrast to the direct estimate of $5 billion when initially asked. Further, the implication is that you believe there is about a 2% chance the true value exceeds $292.5 billion, and about a 2% chance it falls below $94.5 billion. Since the actual value is below that lower limit, there is evidence that you exhibit overconfidence in expressing subjective beliefs regarding objective values.

A Standalone APP

We have developed a standalone app to estimate the pdf and cdf of your subjective probability estimates to any problem by applying 10,000 iterations of a computer simulation of the general lognormal distribution, based on the five-number summary you enter.

Jerry Platt, Ph.D., Emeritus Professor of Finance, San Francisco State U.

Comments 39

  1. What’s Happening i am new to this, I stumbled upon this I’ve found It positively helpful and
    it has helped me out loads. I am hoping to contribute & help different users like
    its helped me. Great job.

  2. Have you ever thought about writing an ebook or guest authoring on other websites?
    I have a blog centered on the same information you discuss and would really like to have you
    share some stories/information. I know my subscribers
    would enjoy your work. If you’re even remotely interested, feel free to send me
    an email.

  3. Howdy! This article could not be written much better!
    Reading through this post reminds me of my previous roommate!
    He always kept preaching about this. I will send this
    information to him. Fairly certain he’s going to have a
    very good read. Thank you for sharing!

  4. İnstagram kullanıcıları daha aktif ve etkin olmasını istiyorsanız takipçi sayınızı belirli bir düzeye ulaştırmalısınız.

    Bunun kendi imkanlarınızla kısa sürelerde
    hallediyor olmanız neredeyse imkansızdır.

    İnstagram takipçi satın al hizmeti ile hesaplarınızda istediğiniz sayılarda takipçi sahibi olabilirsiniz.

    Bu alanda hizmet veren ve takipçi satışı yapan platformları kullanabilirsiniz takipcisatinalin.org
    Instagram Takipçi Satın Al hizmetleriyle aktif bir
    şekilde takipçi hizmeti veriyor

  5. Hey! This post couldn’t be written any better! Reading through this post reminds me of my
    good old room mate! He always kept chatting about this. I will forward this write-up to him.
    Fairly certain he will have a good read. Thanks for sharing!

  6. I’m impressed, I have to admit. Rarely do I encounter a blog that’s both educative and amusing, and without
    a doubt, you have hit the nail on the head. The issue is something which too few men and women are speaking intelligently about.
    Now i’m very happy that I stumbled across this in my hunt for something concerning this.

  7. Its like you read my thoughts! You appear to grasp so much approximately this, such as you wrote the ebook in it or something.
    I believe that you just could do with a few percent to power the message
    house a little bit, however other than that, this is excellent blog.

    A fantastic read. I will certainly be back.

  8. Hi there I am so happy I found your website, I really found you by accident, while I
    was searching on Yahoo for something else, Nonetheless
    I am here now and would just like to say many thanks for a tremendous post
    and a all round thrilling blog (I also love the
    theme/design), I don’t have time to look over it all at the minute but I have saved it
    and also added your RSS feeds, so when I have time I
    will be back to read a great deal more, Please do keep
    up the excellent job.

  9. When I initially left a comment I seem to have clicked the -Notify me when new comments are added- checkbox and from now on whenever
    a comment is added I get four emails with the same comment.

    There has to be a way you are able to remove me from that service?
    Many thanks!

  10. Hello I am so thrilled I found your site, I really found you by mistake, while I was researching on Yahoo for something else, Anyhow I am here now and would just like to say
    cheers for a tremendous post and a all round exciting
    blog (I also love the theme/design), I don’t have time to look
    over it all at the minute but I have saved it and also included your RSS feeds,
    so when I have time I will be back to read a great deal more, Please do keep up the fantastic job.

  11. aşam alanlarımızda sık sık rastladığımız hamam böceği problemleri,
    kısa vakit içinde araya girmek edilmez ise çok daha değişik boyutlarda sorunlara niçin olabilirler.
    antalya böcek ilaçlama Hamam böcekleri dolaştıkları her
    ortamdan bünyelerine detaylı bakteri taşımaktadır.
    Daha sonra bu bakterileri de girmiş oldukları her alanda yayarak, başka
    canlı türlerinin sağlığını rizikoya atabilmektedir.
    Bu amaçla hamam böceği ilaçlama faaliyetlerinin önemi gerektiğince büyüktür.

  12. Greetings, There’s no doubt that your web site could be having web
    browser compatibility problems. When I take a look at your web site in Safari, it
    looks fine however when opening in Internet Explorer,
    it’s got some overlapping issues. I just wanted to provide you with a quick heads
    up! Apart from that, fantastic blog!

  13. instagram gerçek türk beğeni satın al denilince akla hemen ilk gelen site
    olan sitemiz hemen size yılların verdiği
    tecrübe ile sizlere hala kesintisiz hizmet
    vermekte. Türk ve gerçek beğeni satın alabilmek için tercih sebebi
    olan bu siteden hemen faydalanabilirsiniz, Bu sayede profiliniz herkes tarafından sevilecek ve heryerde sizden bahsedecekler.

    Türk beğeni almanın faydalarından biri faydası ise gönderileriniz keşfete düşer.
    Aynı zamanda türk oto beğeni sağlayıcısıdır.
    Oto beğeni siz paylaşım yaptığınız an belirttiğiniz sürede
    paylaşımınız ister foto ister video ne olursa olsun otomatik olarak size ulaştırılır.

  14. Instagram Takipçi Sayısının Önemi
    Günümüzde neredeyse en çok kullanımının olduğu
    uygulama olan instagram uygulamasında kullanıcılar kendilerini
    daha yaygın ve daha yüksek etkileşimler almanın etkisine kaptırmış durumdadır.

    Bu nedenle kişiler daha fazla etkileşim alabilmek adına
    ve daha yaygın bir kitleye ulaşabilme imkanı elde etmek isterler.

    Bu kişilerin takipçi satın al gibi
    platformlar aracılığı ile gerçek takipçiye sahip olarak etkileşimleri en kısa zamanda en yüksek seviyelere taşıma imkanı elde ederler.

  15. I was wondering if you ever considered changing the page layout of your website?
    Its very well written; I love what youve got to say.

    But maybe you could a little more in the way of content so people could connect with it better.
    Youve got an awful lot of text for only having 1 or two pictures.
    Maybe you could space it out better?

  16. It is appropriate time to make some plans for the future and it’s time to be happy.
    I have learn this submit and if I could I wish to counsel you few fascinating issues or suggestions.
    Perhaps you could write subsequent articles relating to this article.
    I wish to learn more issues approximately it!

Leave a Reply

Your email address will not be published. Required fields are marked *