Blog 35: Survivorship Bias by Xavier Lo, FIA, FRM, MBA

Back in World War 2 [第二次世界大戰], the military [軍隊] wanted to see how their war planes [戰機] could be made stronger. They analysed all the planes which returned from missions [任務] and saw that all the bullet holes [子彈窿] were on the bodies of the plane. Hence they concluded that as long as they put more protective armour around the bodies, the planes would be very well protected. This conclusion is a good example of survivorship bias [倖存者偏差], and a good example of where actuaries could save wars [拯救戰爭]!

A mathematician noticed this and concluded the exact opposite. The places which do not have bullet holes, like the wings and engines, are exactly the places where we should put more armour. Why is this? Well, the planes at the base were actually able to return [返回基地] even with the bullet holes. This means that there were probably a lot of planes which got hit in the wings and engines which were not able to return back to the base at all.

Survivorship bias is analysing results based only on successful data points, which I’m sure you can all see will give you very wrong conclusions [錯結論]. For example, you might hear that the average return of very high-risk fund managers [高風險基金經理] can probably be around 20 – 30% per year. In another example, when you read hotel or restaurant reviews, there are some sites where you see amazingly low scores [低評介]. In both examples, I think you can see that there is probably a large portion of data that did not “survive” for the purposes of your analysis (the fund managers who went bankrupt, or the people with positive experience who didn’t comment).

We as actuaries always deal with data analysis. Survivorship bias is something we always have to bear in mind when looking at results. Keep asking yourself: have I missed out any data which would prove my conclusion wrong? We love getting results, but wrong results are more devasting than having no results!

About the Author

Xavier Lo, FIA, FRM, MBA

Qualified fellow actuary (in UK and Hong Kong), Financial Risk Manager, and MBA graduate (listed on the Dean's List) with a passion for insurance, data science, and analytics. Experienced in a broad range of insurance roles (pricing, capital modelling, reserving, ERM), along with a touch of knowledge in banking. Member of the General Insurance Committee (2021), Actuarial Innovation Committee (2019 - 2021) in ASHK.