https://m811.com/Andri Möll2021-02-09T18:05:00.000ZAndri Möllhttps://m811.comhttps://m811.com/articles/id-kaardi-brauserilaienduse-turvalisusest/ID-kaardi brauserilaienduse turvalisusest ja vahendusründest2021-02-09T15:53:09.000Z2021-02-09T18:05:00.000Z<big>
<p>Riigi Infosüsteemi Amet lasi 28. jaanuaril välja uue ID-kaardi tarkvara versiooni, millega parandati veebis ID-kaardiga isikutuvastuse juures vahendusrünne. Vahendusrünne (<a href="https://en.wikipedia.org/wiki/Man-in-the-middle_attack"><em>man-in-the-middle attack</em></a>) seisnes selles, et sina logid sisse ühel veebilehel, kuid telgi taga tehakse sinu nimel toimetusi ka mujal. Nädal hiljem avaldatud <a href="https://www.ria.ee/et/uudised/ria-koos-partneritega-parandas-id-kaardi-brauserilaienduses-kriitilise-vea.html">RIA pressiteadet</a> on mitmed Eesti väljaanded kajastanud, kuid kahjuks ei ole see kogu lugu — turvaviga oli kergem avastada ja ulatub sügavamale kui kirjeldatud.</p>
</big>
<blockquote>
<p>Nõrkus polnud selline, millele võinuks kurjategijad lihtsalt otsa komistada, vaid selle avastamiseks tuli vaeva näha ning teoreetiliseks kuritarvitamiseks oli vaja ka omada kontrolli veebilehe üle, kus saab end ID-kaardiga autentida.
— <a href="https://www.ria.ee/et/uudised/ria-koos-partneritega-parandas-id-kaardi-brauserilaienduses-kriitilise-vea.html">Mark Erlich</a></p>
</blockquote>
<p><strong>Tegelikkuses ei olnud turvaviga haruldaste asjade kokkulangevus ega geniaalne mõttesähvatus</strong>, vaid on olnud <strong>arhitektuuriline viga ID-kaardi brauserilaienduses</strong> loomisest saati. Ka <strong>ei piirdu see autentimisega</strong>, vaid <strong>hõlmab ka allkirjastamist</strong>. Tehniliselt on need kaks protsessi identsed, ent hiljutise kirjavahetuse põhjal ei tundu RIA hetkel sellele tähelepanu pööravat. Võib arvata miks, kuna <strong>allkirjastamise turvavea lahendamine tähendaks kõikide täna ID-kaarti allkirjastamiseks kasutavate veebilehtede ümbertegemist</strong>. Autentimist puudutav <strong>turvaparandus aga oli kehvasti kommunikeeritud</strong>, seejuures <strong>mõistlikku alternatiivi pakkumata</strong> ja siiski <strong>sümptomi, mitte haiguse ravimine</strong>.</p>
<p>Mis puutub vaeva nägemisse, siis <strong>tõenäoliselt on nõrkust märganud mitmed veebiturbega vähegi kursis olevad arendajad</strong>, kes on pidanud ID-kaardi toimetusi veebilehele implementeerima. Ma ei taha hästi uskuda, et see RIAlegi üllatusena tuli, sest paari lõigu pärast märkad viga ka sina, kui detaile kuuled.</p>
<h2 id="id-kaardi-toimimine">ID-kaardi toimimine</h2>
<p>Käime alustuseks ID-kaardi tööpõhimõtted üle, et saaksid paremini aru, kuidas autentimine ja allkirjastamine toimivad ning kuidas vahendusrünne puudutab tegelikult ka allkirjastamist.</p>
<p>ID-kaart tugineb <a href="https://et.wikipedia.org/wiki/Avaliku_v%C3%B5tme_kr%C3%BCptograafia">avaliku võtme krüptograafial</a> (<a href="https://en.wikipedia.org/wiki/Public-key_cryptography">public key cryptography</a>). Võid mõelda sellest nii, et igal ID-kaardil eksisteerib kaks unikaalset numbripaari — üks mõeldud autentimiseks, teine allkirjastamiseks. Mõlemast paarist on üks number teada vaid ID-kaardile ning teine avalikustatud. Salajase numbriga tehtud aritmeetikat saab hiljem avaliku numbriga kontrollida ja veenduda, et tulemus sai tulla vaid salajast numbrit (võtit) teadvalt ID-kaardilt. Kaardi salajasi numbreid kutsutakse vastavalt autentimisvõtmeks ja allkirjastamisvõtmeks ning avalikke numbreid avalikeks võtmeteks. Vaatamata sellele, et ID-kaart isegi omanikule salajasi numbreid (privaatvõtmeid) välja ei ütle, saame paluda tal nendega aritmeetikat teha ning tulemustega teistele tõendada, et tegu on just meie numbripaari ja seega meie ID-kaardiga.</p>
<p>Kuna seadusandluses on reguleeritud, mida emma-kumma võtmepaari (numbripaari) kasutamine juriidiliselt tähendab, saab öelda, et oled end kas tuvastanud või andnud millelegi allkirja. <strong>Siiski on oluline aru saada, et autentimine (mille peale kaart küsib PIN 1) ning allkirjastamine (PIN 2) on tehniliselt ja matemaatiliselt identsed</strong>.</p>
<p>Kui oled varem ID-kaarti kasutanud, oled tõenäoliselt pidanud arvutisse installima RIA arendatud <a href="https://www.id.ee/id-kaart/">ID-tarkvara</a>. Peale DigiDoci rakenduse, kus saad dokumente allkirjastada, võimaldab ID-tarkvara sul ka veebis surfates ID-kaardiga sisse logida või veebis allkirju anda.</p>
<p>Suhtlust ID-kaardi ja veebilehitseja (nt Firefox, Chrome, Internet Explorer vm) vahel vahendab RIA arendatud pistikprogramm (<em>plugin</em>), mis on samasugune lisakomponent brauserile nagu ad-blockerid jm, mida oled ehk ise installinud. Veebilehe autor, kes soovib ID-kaarti toetada, lisab lehele natuke JavaScripti programmeerimiskeeles koodi, mis suhtleb RIA pistikprogrammiga, mis viimaks saadab päringu ID-kaardile. Nii saab küsida ID-kaardilt omaniku andmeid või paluda midagi allkirjastada. ID-kaardiga suhtleva lehe saab veebi paigutada igaüks, kes natukegi JavaScripti ja HTMLi oskab.</p>
<h2 id="autentimine-ja-vahendusrünne">Autentimine ja vahendusrünne</h2>
<p>Järgmiseks vaatame, kuidas ID-kaardiga sisselogimine toimib ja kus vahendusrünne avaldub. Märkad varsti isegi, et erinevalt RIA seisukohast on turvaauk üpris ilmne. Kui oled ID-kaardi tehnilise toimimise ja selle vahendusrünnetega tuttav, võid ka otse <a href="https://m811.com/articles/id-kaardi-brauserilaienduse-turvalisusest/#lahenduse-probleem">RIA lahenduse probleemini</a> liikuda.</p>
<p>Sisselogimise eesmärk on tuvastada, et sinu kontrolli all on eelnevalt mainitud autentimise privaatne võti. Eeldame hetkel, et avalikku võtit, millega aritmeetika tulemust kontrollida, teab veebileht juba varasemast. Tegelikkuses tulevad siin mängu avaliku võtme infrastruktuur koos sertifitseerimisasutustega, aga need pole praegu olulised.</p>
<p>Protsess on üpris lihtne. Veebileht genereerib juhusliku numbri ja palub su ID-kaardil (läbi RIA arendatud pistikprogrammi) autentimisvõtmega seda kinnitada. Miks just juhusliku numbri? Sest vastasel juhul saaks autentimistulemust mujal taaskasutada. Näiteks saaks veebilehe autor su vastusega minna ise hiljem sinu eest mujale sisse logima.</p>
<p>Tehnilisematele inimestele, <a href="https://github.com/hwcrypto/hwcrypto.js">Hwcrypto.js teeki kasutades</a> näeb koodijupp autentimisvõtme kasutamiseks välja selline:</p>
<pre><code class="hljs language-javascript"><span class="hljs-keyword">var</span> signature = Hwcrypto.getCertificate({<span class="hljs-attr">filter</span>: <span class="hljs-string">"AUTH"</span>}).then(<span class="hljs-function"><span class="hljs-keyword">function</span>(<span class="hljs-params">cert</span>) </span>{
Hwcrypto.sign(cert, {<span class="hljs-attr">value</span>: someRandomData, <span class="hljs-attr">type</span>: <span class="hljs-string">"SHA256"</span>}, {})
})
</code></pre>
<p>Selle peale küsib pistikprogramm sinult PIN 1.</p>
<figure class="image">
<p><a href="https://m811.com/articles/id-kaardi-brauserilaienduse-turvalisusest/id-card-pin-1.png"><img src="https://m811.com/articles/id-kaardi-brauserilaienduse-turvalisusest/id-card-pin-1.png" alt="" /></a></p>
<figcaption>
ID-kaardi PIN 1 küsimise aken Chrome brauseris Windowsis.
</figcaption>
</figure>
<p>Kui PIN 1 on õigesti sisestatud, tagastab ID-kaart läbi pistikprogrammi veebilehele matemaatilise tulemuse. Veebileht omakorda kontrollib tulemust su avaliku võtmega ja kui tulemus klapib, saab juriidiliselt öelda, et oled veebilehel end autentinud. Sest see olid ju sina, kes parasjagu sel veebilehel PIN 1 sisestas, eks? Või sai see keegi teine olla?</p>
<p>Mõtleme veebilehe vaatenurgast. Veebileht saab teada soovist sisse logida. Selle peale genereerib leht juhusliku numbri ja annab selle enda meelest külastaja ID-kaardile kinnitamiseks. Kui aga külastaja võtab saadud juhusliku numbri ja ise kinnitamise asemel annab edasi hoopis kellegi teise ID-kaardile kinnitamiseks? Näiteks kellelegi, kes on samal ajal <em>tema</em> kodulehele sisse logimas? Äkki sulle, mõne uue veebipoe külastajale. Autendid end südamerahuga, mõeldes et oled sisenemas e-poodi, samal ajal kui kulisside taga saadab e-pood sinu ID-kaardi kinnituse edasi mõnele teisele lehele, nt internetipanga lehele. Selle peale logitakse hoopis e-poe omanik sinu internetipanka, ilma et oleksid seda takistada saanud. Sellest ka "vahendusrünne" — sina ei saa omalt poolt kontrollida, kuhu tegelikult sisse logid. Probleem on üpris silmnähtav, kas pole, nüüd kus tead, kuidas autentimine töötab? See on ju sama probleem, mis puhkusereisil krediitkaardiga maksmisel — annad kaardi kelnerile, kes sellega tagaruumis peale õhtusöögi arve tasumise ka netist teleri tellib. Pole kaugeltki uut tüüpi avastus.</p>
<p>RIA tegi õigesti klassifitseerides selle turvaauguks. Küll aga tehti seda <strong>etteteatamata</strong>, <strong>mõistliku kasutajasõbraliku alternatiivita</strong> ja <strong>fundamentaalset probleemi lahendamata</strong>. Enne RIA lahenduse käsitlemist, vaatame korra allkirjastamist.</p>
<h2 id="allkirjastamine-ja-vahendusrünne">Allkirjastamine ja vahendusrünne</h2>
<p>Mäletad, et rääkisime, kuidas autentimine ja allkirjastamine on matemaatiliselt identsed, sest ID-kaardil on kaks numbripaari ja need erinevad vaid juriidilise tähenduse poolest? Autentimine ja allkirjastamine on ka tehniliselt identsed. Muutsin eelnevas koodinäites vaid ühe sõna — asendasin sõna <code class="">AUTH</code> sõnaga <code class="">SIGN</code>:</p>
<pre><code class="hljs language-javascript"><span class="hljs-keyword">var</span> signature = Hwcrypto.getCertificate({<span class="hljs-attr">filter</span>: <span class="hljs-string">"SIGN"</span>}).then(<span class="hljs-function"><span class="hljs-keyword">function</span>(<span class="hljs-params">cert</span>) </span>{
Hwcrypto.sign(cert, {<span class="hljs-attr">value</span>: signableHash, <span class="hljs-attr">type</span>: <span class="hljs-string">"SHA256"</span>}, {})
})
</code></pre>
<p>Selle peale küsitakse sinult PIN 1 asemel PIN 2.</p>
<figure class="image">
<p><a href="https://m811.com/articles/id-kaardi-brauserilaienduse-turvalisusest/id-card-pin-2.png"><img src="https://m811.com/articles/id-kaardi-brauserilaienduse-turvalisusest/id-card-pin-2.png" alt="" /></a></p>
<figcaption>
ID-kaardi PIN 2 küsimise aken Chrome brauseris Windowsis.
</figcaption>
</figure>
<p>Tulemus on juriidiliselt võrdne omakäelise allkirjaga. Kuna tehnilised protsessid on võrdsed, võime järeldada, et mis iganes turvavead on autentimisel, puudutavad need ka digiallkirjastamist. Kujuta näiteks ette sama e-poodi, millest just autentimise kontekstis rääkisime. Näiteks võib e-pood sinult paluda järelmaksulepingu allkirjastamist, telgi taga aga sinu eest su allkirjaga kuskil hoopis hääletada. Nagu PIN 2 pildilt ülal näed, ei saa sina kuidagi enne allkirja andmist veenduda, mida päriselt allkirjastad. Kaasav demokraatia on üpris populaarseks muutunud. Miks ei võiks pangad näiteks ülekannet kinnitades su allkirja salaja hoopis mõnda e-demokraatia platvormi suunata? Äkki mõne sellise algatuse toetuseks, mis neile ka kasulik oleks. Ja sina ei saaks sellest kunagi teada, et isegi allkirja tühistama minna.</p>
<p>Üks RIA osakonnajuhataja andis mõista, et allkirjastamise vahendusrünne pole oluline seepärast, et iga allkirjastamist võimaldav e-teenus saab ju niikuinii allkirjastatava dokumendi viimasel hetkel millegi muu vastu välja vahetada. Tõepoolest. Aga <strong>kui ID-kaardi kasutamise turvalisus peakski seisnema veebisaitide usaldamises, siis poleks olnud vaja ka autentimise vahendusründest probleemi teha</strong>. Paneme silmad kinni ja loodame parimat.</p>
<h2 id="lahenduse-probleem">Lahenduse probleem</h2>
<p>Fundamentaalne probleem nii autentimise kui ka allkirjastamisega seisneb selles, et <strong>sina kui ID-kaardi omanik ja veebikülastaja ei tea, mida ID-kaardiga kinnitad</strong>. Nagu ülal kirjeldasin, nii autentimise kui ka digiallkirjastamise puhul jõuab su ID-kaardini vaid üks veebilehe antud number, kuid <strong>numbri allikat ja sihtkohta sa teada ei saa</strong>. Seega pole võimalik öelda, kas kinnitamiseks saadetud numbrit kasutatakse heaotstarbeliselt või hoopis sinu eest salaja mujal sisselogimiseks või allkirjastamiseks. See on nagu <strong>lepingule alla kirjutamine, ilma et sa seda enne lugeda saaks</strong>.</p>
<p>Mida siis RIA vahendusründe takistamiseks tegi? <strong>Keelas pistikprogrammi abil ainult ID-kaardiga autentimise</strong>. Tehniliselt öeldes <a href="https://github.com/open-eid/chrome-token-signing/commit/82d5f5c69c3569916d544998f11fccbab2a77e38">keelas RIA <code class="">AUTH</code> parameetri kasutamise</a> ehk veebilehe ligipääsu autentimisvõtmele. Allkirjastamise vahendusründest pole aga kusagil juttu. Pistikprogrammi koodimuudatus tehti juba 16. detsembril 2020, kuid avalikustati alles 28. jaanuaril <a href="https://github.com/open-eid/chrome-token-signing/commit/01161da77005b185364aa12be88d623ba012715b">GitHubis</a>. Pressiteade avaldati alles nädal hiljem. <strong>Muudatus lõhkus seega üleöö mitmetel saitidel ID-kaardiga autentimise</strong>, sest aastavahetusel anti teada vaid RIA väljavalitutele. Mitmel neist saitidest <strong>ei ole tänaseni ID-kaardiga autentimiseks head alternatiivi</strong>, sest ainus teine meetod, brauseritele sisseehitatud serdiautentimine (<em>TLS client certificate authentication</em>), on tunduvalt kehvema kasutajakogemusega ning paljud serveripakkujad<sup id="fnref-tls-auth"><a href="https://m811.com/articles/id-kaardi-brauserilaienduse-turvalisusest/#fn-tls-auth" class="footnote-ref">tls-auth</a></sup> seda ei toeta. Parema kasutajakogemuse ja võimaluste pärast pistikprogrammiga autentimine leviski. Nagu <a href="https://digi.geenius.ee/eksklusiiv/swedbank-kasutas-turvanorkusega-id-kaardi-laiendust-kaks-aastat-pank-pidas-seda-tookindlamaks/">Swedbankki tunnistas</a>, oli see kasutaja vaatenurgast töökindlam. Allkirjastamise kõrval ei olnud autentimine ohtlikum, sest mõlemad on auklikud.</p>
<p>Tehti õigesti, et probleemiga tegeleti, kuid seda oleks pidanud lahendama teisiti. <strong>Esiteks ei olnud turvaprobleem "peidus" ega vajanud avastamiseks vaeva</strong>. Selliste <strong>märgatavate vigade pea kaks kuud salajas hoidmine teeb rohkem kahju kui head</strong>, sest kõik need, kes on soovinud seda ära kasutada, olid sellest kahtlemata juba aastaid teadlikud. Ainult tavalised veebikasutajad ja veebilehtede omanikud said sellest liiga hilja teada, et end kaitsta.</p>
<p>Teiseks <strong>kulus pea kaks kuud sümptomi eemaldamiseks, mitte fundamentaalse probleemi lahendamiseks</strong>. Olgu, ilmselt parandati sümptomit ehk tunnike, kuid selle avaldamisega viivitati veebruarini. Ja sedagi kirjeldati vaid napisõnaliselt autentimist mainides, olgugi et probleem puudutab ka allkirjastamist. Kahe kuu jooksul oleks saanud autentimise turvaprobleemi lahendada ka ilma selle eemaldamiseta. Eriti kuna niikuinii tuli veebilehtedel muudatusi teha.</p>
<p><strong>Probleemist ja potentsiaalsetest lahendustest oleks pidanud RIA juba detsembris arendajatele avalikult teada andma</strong>. Nõnda oleksid saanud veebiarendajad anda tagasisidet ja ühiselt parema lahenduseni jõuda. Üksikute saitide turvaprobleeme võib suletud uste taga lahendada, ent mis puutub me kõikide ID-kaartide turvalisusesse, siis protsessi läbipaistvus on olulisem. Alustuseks, RIA ei tea kõiki turvaprobleemidest või muudatustest mõjutatud osapooli, sest ID-kaardi toe lisamiseks pole vaja veebilehte kuskil registreerida. Kuid veelgi tähtsam on läbipaistvus seepärast, et <strong>allkirjastamise vahendusründest me nii kergelt lahti ei saa</strong>. Kas või puhtalt seepärast, et see tähendab kõikide täna allkirjastamist toetavate saitide ümbertegemist. Isegi kui tehnilise lahenduse saaks nädalaga välja mõeldud, ei saa sarnaselt autentimisele päevapealt allkirjastamist lõhkuda.</p>
<h2 id="probleemi-lahendamine">Probleemi lahendamine</h2>
<p>Mida siis ette võtta? Kõigepealt leian, et <strong>on oluline inimesi informeerida netis autentimise ja allkirjastamise riskidest</strong>. Riskid puudutavad tegelikult ka Mobiil-ID ja Smart-ID kasutajaid, kuid sellest räägime teinekord. Nii kaua, kui ID-kaardi pistikprogramm ei kuva konkreetselt, kus autendid või mida allkirjastad, <strong>tasub olla ettevaatlik</strong>. Miski ei garanteeri, et telgi taga ei toimu vahendusrünnet ja tõenäoliselt jääb see nii veel pikka aega.</p>
<p>Lohutuseks võin öelda, et brauseritele sisseehitatud sertifikaadiga autentimine, mida RIA autentimiseks kasutada soovitab ja mille kehva kasutajakogemust mainisin, näitab korra saidi aadressi (pildil <code class="">test-eid.eesti.ee</code>). See annab mõista, kuhu sisse logid.</p>
<figure class="image">
<p><a href="https://m811.com/articles/id-kaardi-brauserilaienduse-turvalisusest/id-card-select-certificate.png"><img src="https://m811.com/articles/id-kaardi-brauserilaienduse-turvalisusest/id-card-select-certificate.png" alt="" /></a></p>
<figcaption>
ID-kaardi sertifikaadi valimise aken brauseritele sisseehitatud autentimiseks.
</figcaption>
</figure>
<p>Saiti mainitakse küll vaid sertifikaati valides ja mitte hiljem PIN 1 sisestades, nii et eriti selgeks lahenduseks seda kutsuda ei saa. Õnneks on tehniliselt selline autentimine vahendusründe eest kaitstud, kuid meetod pole kaugeltki töökindel ega kasutajasõbralik. Kirjutan sellest tulevikus, et saaksid ka ise naerda või nutta.</p>
<p><strong>RIAl tuleks oma lähenemine antud probleemi lahendamisel kriitiliselt üle vaadata</strong>. Kui nad ei teadnud, miks veebilehed pistikprogrammiga autentimist teostasid, tasub välja selgitada, mitte naljatada, et <q><a href="https://digi.geenius.ee/eksklusiiv/swedbank-kasutas-turvanorkusega-id-kaardi-laiendust-kaks-aastat-pank-pidas-seda-tookindlamaks/">Põhimõtteliselt avastati, et taignarulliga saab ka naelu seina lüüa</a></q>. Kasutajakogemus ning töökindlus on veebis päris olulised ning sellest tulenevalt on vahel väikese riskiga lahenduste kasutamine teadlik valik. Mõne süsteemi turvalisus sõltub jällegi tema nõrgimast lülist. Kuna allkirjastamine on jätkuvalt ebaturvaline, ei pruugi abi olla ka autentimise kaitsmisest.</p>
<p>Irooniliselt pole RIA enda juhendid aastaid vahendusründele õieti tähelepanu pööranud. Sellest tulenevalt võib kahtlustada, et <strong>lehti, kuhu saab vahendusründega sinu nimel sisse logida, on sadu</strong>. Olukorras, kus autentimise arendamine on niivõrd veaohtlik, tuleks <strong>RIAl võtta vastutus terviksüsteemi</strong> eest, mitte näidata näpuga vaid veebilehtede suunas. Suhtumine, mis seab kogu vastutuse veebilehele, ei ole jätkusuutlik ega praktikas turvaline. <strong>Sinu ID-kaardi ja digiidentiteedi kasutamise turvalisus ei tohi tugineda sellel, et kõik teised veebilehed, k.a teistes riikides, toimivad 100% turvaliselt ja on 100% usaldusväärsed</strong>. Seda ei juhtu kunagi. <strong>Sinul peaks olema võimalus autentides ja allkirjastades kindel olla, kuhu või millele sinu kinnitus lõpuks läheb</strong>.</p>
<p>Sellise <strong>turvalise tehnilise lahenduse väljatöötamine peaks toimuma avalikkuse silme ees</strong>, nii et iga asjast huvitatu saab protsessi jälgida, vajadusel kaasa rääkida ning probleeme varakult välja tuua. ID-kaardi turvaline ja töökindel toimimine on täna hindamatu nii eraelus, äris kui ka demokraatias. <strong>Samade suletud uste taga jõuame sama tulemuseni.</strong></p>
<hr />
<p>Kui sul on artikli sisu kohta küsimusi, võid kirjutada <a href="https://m811.com/articles/id-kaardi-brauserilaienduse-turvalisusest/mailto:andri@dot.ee">andri@dot.ee</a>.</p>
<div class="footnotes">
<hr />
<ol>
<li id="fn-tls-auth">Levinud rakenduste majutusplatvormid nagu Heroku jt ei toeta sertifikaatidega autentimist.<a href="https://m811.com/articles/id-kaardi-brauserilaienduse-turvalisusest/#fnref-tls-auth" class="footnote-backref">↩</a></li>
</ol>
</div>https://m811.com/articles/barotrauma-from-source/Compiling Barotrauma from Source2020-05-19T15:26:21.000Z2020-05-19T15:26:04.000Z<big>
<p><a href="https://barotraumagame.com">Barotrauma</a> is a 2D co-op survival game about navigating the depths of the Jupiter's moon Europa in a submarine. With the help of other players or NPCs, you need to operate the submarine, maintain the onboard systems, repair leaks, fight the local fish, take care of the crew and stay alive. It's sold on <a href="https://store.steampowered.com/app/602960/Barotrauma">Steam</a> and runs on Linux. What's better, its <a href="https://github.com/Regalis11/Barotrauma">source code is available on GitHub</a> for modders and limited joint development. Here's a guide on how to compile Barotrauma on Linux <em>for</em> Linux.</p>
</big>
<figure class="image">
<p><a href="https://m811.com/articles/barotrauma-from-source/screenshot.png"><img src="https://m811.com/articles/barotrauma-from-source/screenshot.png" alt="" /></a></p>
<figcaption>
In-game screenshot.
</figcaption>
</figure>
<p>While the source for the game is publicly available, its graphical and other assets are not. To get a playable game at the end, you'll need to <a href="https://store.steampowered.com/app/602960/Barotrauma">purchase Barotrauma on Steam</a> both for its <code class="">Content</code> directory containing its assets and multiplayer, which depends on Steam's match-making. Compiling from source as-is won't net you a <a href="https://en.wikipedia.org/wiki/Digital_rights_management">DRM</a>-free game. If that's your goal, you'll have to figure it out yourself, something the developer warns against in their <a href="https://github.com/Regalis11/Barotrauma/blob/master/EULA.txt">end-user license agreement</a>.</p>
<p>That out of the way, let's begin.</p>
<h2 id="setup">Setup</h2>
<p>These instructions are written for compiling Barotrauma for Linux <em>on</em> Arch Linux. However, I suspect they'll apply equally well to other platforms, as beyond a few package mentions, there's nothing specific to Arch Linux. You're going to need some familiarity with the command line and developer tools. Nothing you couldn't learn now.</p>
<p>First, ensure you've got <a href="https://git-scm.com/">Git</a> installed.</p>
<pre><code class="hljs language-sh terminal">pacman -S git
</code></pre>
<p>Then, let's clone the <a href="https://github.com/Regalis11/Barotrauma">Barotrauma's repository from Github</a>. To save disk space, let's do a shallow clone that skips the history and just grabs the latest state. As of May 17, 2020, that'll be <a href="https://github.com/Regalis11/Barotrauma/commit/ce4ccd99acf240cea3c0bdee88d01d78173e5192">ce4ccd9</a>, which corresponds to v0.9.9.1. In total, 138.3 MiB.</p>
<pre><code class="hljs language-sh terminal">git <span class="hljs-built_in">clone</span> --depth 1 https://github.com/Regalis11/Barotrauma.git
</code></pre>
<p>The game's written in C# using the <a href="https://dotnet.microsoft.com/download">.NET Core framework</a>. We'll need its SDK installed to run the build scripts and compile the game. Arch Linux has .NET Core available in its official repositories, making the installation easy. Barotrauma's <a href="https://github.com/Regalis11/Barotrauma">README</a> mentions .NET Core v3.0, but v3.1 seems to work fine.</p>
<pre><code class="hljs language-sh terminal">pacman -S dotnet-sdk
</code></pre>
<p>The rest of the game's dependencies will either be installed by the .NET Core <code class="">dotnet</code> executable or are committed to the source repository as binary files. If I were you, I'd also now prepare an unprivileged user account or some other sandboxing to run the build under. After all, you'll be running a lot of unaudited code off the Internet. One relatively convenient sandbox utility on Linux is <a href="https://firejail.wordpress.com/">Firejail</a> (<a href="https://wiki.archlinux.org/index.php/firejail">Firejail on Arch Wiki</a>).</p>
<h2 id="compiling">Compiling</h2>
<p>The primary building and packaging happens via <code class="">dotnet</code>, a build tool that came with the .NET Core SDK. To get the Barotrauma Linux client to successfully compile, though, we'll need to download a missing controller mapping file and apply a patch to skip copying already bundled dependencies.</p>
<h3 id="sdl-game-controller-mappings">SDL Game Controller Mappings</h3>
<p>For some reason the build depends, but doesn't automatically download, the game controller mapping database for <a href="http://www.libsdl.org">SDL</a> from <a href="https://github.com/gabomdq/SDL_GameControllerDB">SDL_GameControllerDB</a>. Let's do so ourselves.</p>
<pre><code class="hljs language-sh terminal">mkdir -p Libraries/MonoGame.Framework/Src/ThirdParty/SDL_GameControllerDB
wget https://raw.githubusercontent.com/gabomdq/SDL_GameControllerDB/master/gamecontrollerdb.txt -O Libraries/MonoGame.Framework/Src/ThirdParty/SDL_GameControllerDB/gamecontrollerdb.txt
</code></pre>
<p>Those that try to run the <code class="">dotnet publish</code> command from below before downloading the controller configuration file, will encounter the following error:</p>
<pre><code class="">CSC : error CS1566: Error reading resource 'gamecontrollerdb.txt' -- 'Could not find a part of the path '/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/ThirdParty/SDL_GameControllerDB/gamecontrollerdb.txt'.' [/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/MonoGame.Framework/MonoGame.Framework.Linux.NetStandard.csproj]
</code></pre>
<h3 id="skipping-compiling-bundled-dependencies">Skipping Compiling Bundled Dependencies</h3>
<p>As can be guessed from its use of the <a href="https://github.com/gabomdq/SDL_GameControllerDB">SDL game controller database</a>, Barotrauma depends on <a href="http://www.libsdl.org">SDL</a>. It also depends on <a href="https://www.openal.org/">OpenAL</a> for audio. Actually, Barotrauma depends on the <a href="https://www.monogame.net">MonoGame framework</a>, which is what seems to depend on both. Nonetheless, MonoGame seems to insist on having access to SDL's and OpenAL's dynamic libraries for all three platforms — Windows, Linux and Mac — during its build. Fortunately for us, Barotrauma's developers have included compiled versions of the former in the repository. As MonoGame is still looking for them in the <code class="">ThirdParty</code> directory, though, we'll need to patch the build project in <a href="https://github.com/Regalis11/Barotrauma/blob/ce4ccd99acf240cea3c0bdee88d01d78173e5192/Libraries/MonoGame.Framework/Src/MonoGame.Framework/MonoGame.Framework.Linux.NetStandard.csproj"><code class="">Libraries/MonoGame.Framework/Src/MonoGame.Framework/MonoGame.Framework.Linux.NetStandard.csproj</code></a> to not do so.</p>
<p>Copy the diff below to a file or just <a href="https://m811.com/articles/barotrauma-from-source/monogame-ce4ccd9.patch" download>download the patch</a> directly.</p>
<pre><code class="hljs language-patch"><span class="hljs-comment">diff --git a/Libraries/MonoGame.Framework/Src/MonoGame.Framework/MonoGame.Framework.Linux.NetStandard.csproj b/Libraries/MonoGame.Framework/Src/MonoGame.Framework/MonoGame.Framework.Linux.NetStandard.csproj</span>
<span class="hljs-comment">index 7ef537f..63c1b2a 100644</span>
<span class="hljs-comment">--- a/Libraries/MonoGame.Framework/Src/MonoGame.Framework/MonoGame.Framework.Linux.NetStandard.csproj</span>
<span class="hljs-comment">+++ b/Libraries/MonoGame.Framework/Src/MonoGame.Framework/MonoGame.Framework.Linux.NetStandard.csproj</span>
<span class="hljs-meta">@@ -511,64 +511,6 @@</span>
</Compile>
</ItemGroup>
<span class="hljs-deletion">- <ItemGroup></span>
<span class="hljs-deletion">- <None Include="..\ThirdParty\Dependencies\MonoGame.Framework.dll.config"></span>
<span class="hljs-deletion">- <Platforms>WindowsGL,Linux,MacOS</Platforms></span>
<span class="hljs-deletion">- <Link>MonoGame.Framework.dll.config</Link></span>
<span class="hljs-deletion">- <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory></span>
<span class="hljs-deletion">- </None></span>
<span class="hljs-deletion">- <None Include="..\ThirdParty\Dependencies\SDL\Windows\x86\SDL2.dll"></span>
<span class="hljs-deletion">- <Platforms>WindowsGL,Linux</Platforms></span>
<span class="hljs-deletion">- <Link>x86\SDL2.dll</Link></span>
<span class="hljs-deletion">- <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory></span>
<span class="hljs-deletion">- </None></span>
<span class="hljs-deletion">- <None Include="..\ThirdParty\Dependencies\SDL\Windows\x64\SDL2.dll"></span>
<span class="hljs-deletion">- <Platforms>WindowsGL,Linux</Platforms></span>
<span class="hljs-deletion">- <Link>x64\SDL2.dll</Link></span>
<span class="hljs-deletion">- <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory></span>
<span class="hljs-deletion">- </None></span>
<span class="hljs-deletion">- <None Include="..\ThirdParty\Dependencies\openal-soft\Windows\x86\soft_oal.dll"></span>
<span class="hljs-deletion">- <Platforms>WindowsGL,Linux</Platforms></span>
<span class="hljs-deletion">- <Link>x86\soft_oal.dll</Link></span>
<span class="hljs-deletion">- <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory></span>
<span class="hljs-deletion">- </None></span>
<span class="hljs-deletion">- <None Include="..\ThirdParty\Dependencies\openal-soft\Windows\x64\soft_oal.dll"></span>
<span class="hljs-deletion">- <Platforms>WindowsGL,Linux</Platforms></span>
<span class="hljs-deletion">- <Link>x64\soft_oal.dll</Link></span>
<span class="hljs-deletion">- <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory></span>
<span class="hljs-deletion">- </None></span>
<span class="hljs-deletion">- <None Include="..\ThirdParty\Dependencies\SDL\Linux\x86\libSDL2-2.0.so.0"></span>
<span class="hljs-deletion">- <Platforms>WindowsGL,Linux</Platforms></span>
<span class="hljs-deletion">- <Link>x86\libSDL2-2.0.so.0</Link></span>
<span class="hljs-deletion">- <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory></span>
<span class="hljs-deletion">- </None></span>
<span class="hljs-deletion">- <None Include="..\ThirdParty\Dependencies\SDL\Linux\x64\libSDL2-2.0.so.0"></span>
<span class="hljs-deletion">- <Platforms>WindowsGL,Linux</Platforms></span>
<span class="hljs-deletion">- <Link>x64\libSDL2-2.0.so.0</Link></span>
<span class="hljs-deletion">- <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory></span>
<span class="hljs-deletion">- </None></span>
<span class="hljs-deletion">- <None Include="..\ThirdParty\Dependencies\openal-soft\Linux\x86\libopenal.so.1"></span>
<span class="hljs-deletion">- <Platforms>WindowsGL,Linux</Platforms></span>
<span class="hljs-deletion">- <Link>x86\libopenal.so.1</Link></span>
<span class="hljs-deletion">- <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory></span>
<span class="hljs-deletion">- </None></span>
<span class="hljs-deletion">- <None Include="..\ThirdParty\Dependencies\openal-soft\Linux\x64\libopenal.so.1"></span>
<span class="hljs-deletion">- <Platforms>WindowsGL,Linux</Platforms></span>
<span class="hljs-deletion">- <Link>x64\libopenal.so.1</Link></span>
<span class="hljs-deletion">- <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory></span>
<span class="hljs-deletion">- </None></span>
<span class="hljs-deletion">- <None Include="..\ThirdParty\Dependencies\SDL\MacOS\Universal\libSDL2-2.0.0.dylib"></span>
<span class="hljs-deletion">- <Platforms>WindowsGL,Linux,MacOS</Platforms></span>
<span class="hljs-deletion">- <Link>libSDL2-2.0.0.dylib</Link></span>
<span class="hljs-deletion">- <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory></span>
<span class="hljs-deletion">- </None></span>
<span class="hljs-deletion">- <None Include="..\ThirdParty\Dependencies\openal-soft\MacOS\Universal\libopenal.1.dylib"></span>
<span class="hljs-deletion">- <Platforms>WindowsGL,Linux,MacOS</Platforms></span>
<span class="hljs-deletion">- <Link>libopenal.1.dylib</Link></span>
<span class="hljs-deletion">- <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory></span>
<span class="hljs-deletion">- </None></span>
<span class="hljs-deletion">- </ItemGroup></span>
<span class="hljs-deletion">-</span>
<ItemGroup>
<EmbeddedResource Include="MonoGame.bmp">
<LogicalName>MonoGame.bmp</LogicalName>
</code></pre>
<p>Then navigate to the source directory and pass the patch to the <code class="">patch</code> executable. In case you're missing <code class="">patch</code>, install it via Pacman (<code class="">pacman -S patch</code>).</p>
<pre><code class="hljs language-sh terminal">patch -p1 < monogame-ce4ccd9.patch
</code></pre>
<p>You could verify the patch applied correctly by checking Git.</p>
<pre><code class="hljs language-sh terminal">$ git status --short
M Libraries/MonoGame.Framework/Src/MonoGame.Framework/MonoGame.Framework.Linux.NetStandard.csproj
</code></pre>
<p>Skipping patching the MonoGame build project before attempting to build Barotrauma results in the following errors referencing DLLs from Windows, Linux and Mac, even if you're building only for a Linux system.</p>
<details>
<summary>Barotrauma's MonoGame compile errors</summary>
<pre><code class="">/usr/share/dotnet/sdk/3.1.103/Microsoft.Common.CurrentVersion.targets(4570,5): error MSB3030: Could not copy the file "/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/ThirdParty/Dependencies/MonoGame.Framework.dll.config" because it was not found. [/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/MonoGame.Framework/MonoGame.Framework.Linux.NetStandard.csproj]
/usr/share/dotnet/sdk/3.1.103/Microsoft.Common.CurrentVersion.targets(4570,5): error MSB3030: Could not copy the file "/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/ThirdParty/Dependencies/SDL/Linux/x64/libSDL2-2.0.so.0" because it was not found. [/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/MonoGame.Framework/MonoGame.Framework.Linux.NetStandard.csproj]
/usr/share/dotnet/sdk/3.1.103/Microsoft.Common.CurrentVersion.targets(4570,5): error MSB3030: Could not copy the file "/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/ThirdParty/Dependencies/SDL/Linux/x86/libSDL2-2.0.so.0" because it was not found. [/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/MonoGame.Framework/MonoGame.Framework.Linux.NetStandard.csproj]
/usr/share/dotnet/sdk/3.1.103/Microsoft.Common.CurrentVersion.targets(4570,5): error MSB3030: Could not copy the file "/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/ThirdParty/Dependencies/SDL/MacOS/Universal/libSDL2-2.0.0.dylib" because it was not found. [/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/MonoGame.Framework/MonoGame.Framework.Linux.NetStandard.csproj]
/usr/share/dotnet/sdk/3.1.103/Microsoft.Common.CurrentVersion.targets(4570,5): error MSB3030: Could not copy the file "/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/ThirdParty/Dependencies/SDL/Windows/x64/SDL2.dll" because it was not found. [/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/MonoGame.Framework/MonoGame.Framework.Linux.NetStandard.csproj]
/usr/share/dotnet/sdk/3.1.103/Microsoft.Common.CurrentVersion.targets(4570,5): error MSB3030: Could not copy the file "/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/ThirdParty/Dependencies/SDL/Windows/x86/SDL2.dll" because it was not found. [/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/MonoGame.Framework/MonoGame.Framework.Linux.NetStandard.csproj]
/usr/share/dotnet/sdk/3.1.103/Microsoft.Common.CurrentVersion.targets(4570,5): error MSB3030: Could not copy the file "/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/ThirdParty/Dependencies/openal-soft/Linux/x64/libopenal.so.1" because it was not found. [/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/MonoGame.Framework/MonoGame.Framework.Linux.NetStandard.csproj]
/usr/share/dotnet/sdk/3.1.103/Microsoft.Common.CurrentVersion.targets(4570,5): error MSB3030: Could not copy the file "/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/ThirdParty/Dependencies/openal-soft/Linux/x86/libopenal.so.1" because it was not found. [/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/MonoGame.Framework/MonoGame.Framework.Linux.NetStandard.csproj]
/usr/share/dotnet/sdk/3.1.103/Microsoft.Common.CurrentVersion.targets(4570,5): error MSB3030: Could not copy the file "/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/ThirdParty/Dependencies/openal-soft/MacOS/Universal/libopenal.1.dylib" because it was not found. [/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/MonoGame.Framework/MonoGame.Framework.Linux.NetStandard.csproj]
/usr/share/dotnet/sdk/3.1.103/Microsoft.Common.CurrentVersion.targets(4570,5): error MSB3030: Could not copy the file "/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/ThirdParty/Dependencies/openal-soft/Windows/x64/soft_oal.dll" because it was not found. [/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/MonoGame.Framework/MonoGame.Framework.Linux.NetStandard.csproj]
/usr/share/dotnet/sdk/3.1.103/Microsoft.Common.CurrentVersion.targets(4570,5): error MSB3030: Could not copy the file "/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/ThirdParty/Dependencies/openal-soft/Windows/x86/soft_oal.dll" because it was not found. [/home/user/Barotrauma/Libraries/MonoGame.Framework/Src/MonoGame.Framework/MonoGame.Framework.Linux.NetStandard.csproj]
</code></pre>
</details>
<h3 id="compiling-the-game">Compiling the Game</h3>
<p>With the SDL game controller database downloaded and MonoGame's build project patched, we're ready to build Barotrauma. For that, we'll run <code class="">dotnet publish</code> and pass it the project file for the Linux game:</p>
<pre><code class="hljs language-sh terminal">dotnet publish Barotrauma/BarotraumaClient/LinuxClient.csproj --configuration Release --self-contained --runtime linux-x64 /p:Platform=x64 /p:DebugType=None --output game
</code></pre>
<p>Depending on the performance of your computer, this should take 10–20 seconds. You'll find the executable files for the game in the <code class="">game</code> directory (as requested by <code class="">--output game</code>). As of May 17, 2020, it generates around 133.2 MiB worth of files. All that's missing now are the graphical, audio and other assets. Locate the game's directory in your Steam library and copy the <code class="">Content</code> directory over to <code class="">game</code>. The only conflicts you should see are <code class="">*.xnb</code> files in <code class="">Content/Effects</code>. Skip overwriting those.</p>
<p>That's it. Copy the <code class="">game</code> directory where you see fit and run the <code class="">Barotrauma</code> executable.</p>
<pre><code class="hljs language-sh terminal">./Barotrauma
</code></pre>
<p>Your game preferences are in the <code class="">config_player.xml</code> file in the game directory. If you wish to restore your video and control preferences, copy it from Steam, too. The <code class="">config_player.xml</code> file is worth taking a look also for disabling the splash screen, analytics reporting and possible other tweaks. Barotrauma's saves are in <code class="">~/.local/share/Daedalic Entertainment GmbH/Barotrauma</code>, so those should be available wherever you put the compiled game.</p>
<p>By the way, should the command line invocation above not work by the time you're reading this, you could also review the <a href="https://github.com/Regalis11/Barotrauma/tree/master/Deploy/Linux"><code class="">Deploy/Linux</code></a> directory in the repository. It includes the scripts the game authors use for compiling.</p>
<h3 id="compiling-the-server">Compiling the Server</h3>
<p>If you intend to host a server with your compiled version, you'll also need to compile the server executable. Using the same <code class="">dotnet publish</code> invocation as before, replace <code class="">Client</code> with <code class="">Server</code>:</p>
<pre><code class="hljs language-sh terminal">dotnet publish Barotrauma/BarotraumaServer/LinuxServer.csproj --configuration Release --self-contained --runtime linux-x64 /p:Platform=x64 /p:DebugType=None --output game
</code></pre>
<p>This adds the <code class="">DedicatedServer</code> executable (along with a DLL) to the <code class="">game</code> output folder.</p>
<h3 id="build-options">Build Options</h3>
<p>The <code class="">dotnet</code> build system takes a number of options. For the sake of reference, here are the options used above as described by <a href="https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-publish"><code class="">dotnet publish --help</code></a>:</p>
<table><thead><tr><th>Option</th><th>Description</th></tr></thead><tbody><tr><td><code class="">--configuration</code></td><td>The configuration to publish for. The default for most projects is 'Debug'.</td></tr><tr><td><code class="">--self-contained</code></td><td>Publish the .NET Core runtime with your application so the runtime doesn't need to be installed on the target machine.</td></tr><tr><td><code class="">--runtime</code></td><td>The target runtime to publish for. This is used when creating a self-contained deployment.</td></tr><tr><td><code class="">/p:DebugType=None </code></td><td>Skips generating and copying over debug symbol files (<code class="">*.pdb</code>).</td></tr></tbody></table>
<p>If you're playing on the same machine you compiled on (and have .NET Core installed), you could save a bit of disk space by not bundling the .NET Core libraries in the game output folder. Pass <code class="">--no-self-contained</code> to <code class="">dotnet</code> to reduce the game directory to 55.2 MiB. I think it's safer to keep them bundled, should you want to copy the game over to another computer or just uninstall the <code class="">dotnet-sdk</code> package previously installed.</p>
https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/Notes on «COVID-19 — An Initial View» talk2020-05-07T21:50:21.000Z2020-05-09T10:23:38.000Z<big>
<p>Chris Whitty presents the then-current understanding of the <a href="https://en.wikipedia.org/wiki/Coronavirus_disease_2019">2019–2020 coronavirus pandemic</a>. It covers its epidemiology and a few related historical epidemics. He presents a simple systematic approach to talking about viral epidemics, covering disease mortality, transmission and ends with possible treatments and vaccines.</p>
</big>
<h2 id="covid-19">COVID-19</h2>
<p>The virus "SARS-CoV-2" originates from the <a href="https://en.wikipedia.org/wiki/Hubei">Hubei province</a> in China around the end of 2019. Its initial carriers were probably <a href="https://en.wikipedia.org/wiki/Horseshoe_bat">horseshoe bats</a>, but since then it's become a human-to-human disease. It utilizes a common receptor in the cells of the human upper respiratory tract, <a href="https://en.wikipedia.org/wiki/Angiotensin-converting_enzyme_2">ACE-2</a>, to gain entry. It ends up causing COVID-19, a respiratory disease. For mild or early COVID-19, this includes the following symptoms.</p>
<ul>
<li>Fever</li>
<li>Cough</li>
<li>Myalgia (muscle aches)</li>
<li>Headaches</li>
<li>Shortness of breath</li>
<li>Chills</li>
<li>Sore throat</li>
<li>Losing sense of smell or taste</li>
</ul>
<p>Majority of symptomatic people have a fever or a cough. They don't need to seek medical assistance unless they deteriorate. They must, however, self-isolate. Most recover within 7 days, while the cough may remain for a few more weeks. More severe cases have respiratory failure, requiring oxygen. Organ failure is probably an immunological reaction and not directly caused by the virus.
Critical cases have rapid deterioration. Risk of clots like pulmonary embolism and strokes. 25% of people requiring ventilation also require renal support due to kidney or multi-organ disease.</p>
<h3 id="spread">Spread</h3>
<p>Its initial spread was very quick. In a matter of 43 days it moved from 14 counties to 1386 counties in China.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/china-provinces-spread.png"><img src="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/china-provinces-spread.png" alt="COVID-19 spread through China's provinces" /></a></p>
<figcaption>
<p>Spread of COVID-19 from Dec 31, 2019 to Feb 11, 2020. From "<a href="https://cdn.onb.it/2020/03/COVID-19.pdf.pdf">The Epidemiological Characteristics of an Outbreak of 2019 Novel Coronavirus Diseases (COVID-19) — China, 2020</a>".</p>
</figcaption>
</figure>
<p>Initially few outbreaks beyond China, but by March 11 it was declared a pandemic by the World Health Organization. Today, as of May 7, 2020, its world spread looks like:</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/total-confirmed-cases-of-covid-19-per-million-people.svg"><img src="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/total-confirmed-cases-of-covid-19-per-million-people.svg" alt="Total confirmed COVID-19 cases world map" /></a></p>
<figcaption>
<p>From <a href="https://ourworldindata.org/grapher/total-confirmed-cases-of-covid-19-per-million-people?year=2020-05-07">Our World in Data confirmed COVID-19 cases map</a>.</p>
</figcaption>
</figure>
<h3 id="testing">Testing</h3>
<p>As the genome of the virus was found early, a <strong>direct virus test</strong> was rapidly developed utilizing the <a href="https://en.wikipedia.org/wiki/Polymerase_chain_reaction">PCR</a> method. This permits identifying infected people only while they're infected.</p>
<p>Alternatively there are <strong>serology tests</strong>, which identify antibodies. These help find out how many people have had the disease, even those lacking symptoms. It's unknown whether having SARS-CoV-2 antibodies implies immunity or lack of future infectiousness. There are some reports of people having been reinfected. The duration of possibly immunity is therefore unknown. There are multiple different antibodies, some of which are discoverable only 3–4 weeks after the disease.</p>
<p>For larger testing efforts the question of sampling strategies needs resolving, especially regarding asymptomatic cases. Faster tests are still needed to quickly identify people coming to hospitals.</p>
<h3 id="systematic-approach">Systematic Approach</h3>
<p>Whitty presents a system approach to talking about epidemics:</p>
<ul>
<li><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/#mortality">Mortality and severity (virulence)</a></li>
<li><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/#route-of-transmission">Route of transmission</a></li>
<li><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/#force-of-transmission">Force of transmission</a></li>
<li><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/#treatments-and-vaccines">Treatments and vaccines</a></li>
</ul>
<p>Out of the 5, we know we lack available treatments and vaccines for COVID-19.</p>
<h2 id="mortality">Mortality</h2>
<p>Mortality is determined by how many people get infected and of those, how many die (fatality rate). Infection fatality rate (IFR) for COVID-19 seems to be around or below 1%. This could be adjusted once we discover how many people are asymptomatic, infected without knowing it. Compared to pandemics below, it's fairly low:</p>
<table><thead><tr><th>Disease</th><th>Infection fatality rate</th></tr></thead><tbody><tr><td>Ebola</td><td>70%</td></tr><tr><td>HIV</td><td>100% when first emerged</td></tr><tr><td>Smallpox</td><td>30%</td></tr><tr><td><a href="https://en.wikipedia.org/wiki/2009_swine_flu_pandemic">Swine flu from 2009</a> (H1N1)</td><td>0.1%</td></tr><tr><td><a href="https://en.wikipedia.org/wiki/Spanish_flu">Spanish flu from 1918</a> (H1N1)</td><td>~3%</td></tr></tbody></table>
<p>However, even a disease with low mortality, if it infects large numbers of people, ends up killing more people than diseases with higher fatality rates.</p>
<p>Initially, COVID-19s case fatality rate (chance of dying if you do develop symptoms as opposed to merely being infected) was high. That's often to be expected, as with new diseases people tend to notice deaths and severe cases. Only later will milder cases be included. As can be seen from the graph below, the assumed mortality changed over time as more cases were identified.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/reported-deaths-to-reported-cases-2020-02-04-who.jpg"><img src="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/reported-deaths-to-reported-cases-2020-02-04-who.jpg" alt="Graph of reported deaths to reported cases" /></a></p>
<figcaption>
<p>Assumed mortality from <a href="https://www.who.int/docs/default-source/coronaviruse/who-china-joint-mission-on-covid-19-final-report.pdf">World Health Organization joint mission to China report</a></p>
</figcaption>
</figure>
<h3 id="other-coronaviruses">Other Coronaviruses</h3>
<p>Up to 15% of ordinary colds and the occasional pneumonia are caused by 4 other coronaviruses:</p>
<ul>
<li>229E (alpha coronavirus)</li>
<li>NL63 (alpha coronavirus)</li>
<li>OC43 (beta coronavirus)</li>
<li>HKU1 (beta coronavirus)</li>
</ul>
<p>Over the years, two others have arisen:</p>
<ul>
<li><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/#mers">MERS</a> (fatality ~35%)</li>
<li><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/#sars">SARS</a> (fatality 11–15%)</li>
</ul>
<h3 id="risk-of-death">Risk of Death</h3>
<p>Individually the chance of dying from COVID-19 are low.</p>
<ul>
<li>A high proportion of people will not get it.</li>
<li>A significant proportion of people will have no symptoms.</li>
<li>Great majority (~80%) of <strong>symptomatic</strong> cases develop to a mild to moderate disease.</li>
<li>A minority need hospitalization, mostly only oxygen. Majority will survive.</li>
<li>Minority of the prior will need ventilation.</li>
<li>Minority of every age-group die, however even in a risk-group majority do not.</li>
</ul>
<p>Risk factors could be categories into three — age, gender and comorbidities.</p>
<h3 id="age">Age</h3>
<p>The biggest risk factor is age. The case fatality rate (chance of dying if you develop <strong>symptoms</strong>) of all age groups is below.</p>
<figure class="table">
<table><thead><tr><th>Age group</th><th>Case fatality rate</th></tr></thead><tbody><tr><td>0-9</td><td>0.003%</td></tr><tr><td>10-19</td><td>0.01%</td></tr><tr><td>20-29</td><td>0.06%</td></tr><tr><td>30-39</td><td>0.15%</td></tr><tr><td>40-49</td><td>0.3%</td></tr><tr><td>50-59</td><td>1.25%</td></tr><tr><td>60-69</td><td>4%</td></tr><tr><td>70-79</td><td>8.6%</td></tr><tr><td>80+</td><td>13.4% (IFR 7%?)</td></tr></tbody></table>
<figcaption>
<p>From "<a href="https://www.thelancet.com/pdfs/journals/laninf/PIIS1473-3099(20)30243-7.pdf">Estimates of the severity of coronavirus disease 2019: a model-based analysis</a>", with <a href="https://github.com/mrc-ide/COVID19_CFR_submission">replication data on GitHub</a> and <a href="https://www.thelancet.com/cms/10.1016/S1473-3099(20)30243-7/attachment/55646cff-fc65-47d9-a909-38d691cdf9f9/mmc1.pdf">supplementary appendix</a>.</p>
</figcaption>
</figure>
<p>The numbers above seem to match data from other countries, such as England and Wales, Italy and Spain. You see very few deaths among those below 50.</p>
<p>Keep in mind that the <strong>infection fatality rate</strong>, chance of dying if you're infected, could differ from <strong>case fatality rate</strong>, the chance of dying if you develop symptoms. The ratio for COVID-19 is still unknown.</p>
<p>Older people are generally far more likely to die of infection than other age groups, although that doesn't hold universally. The <a href="https://en.wikipedia.org/wiki/Spanish_flu">1918 Spanish flu</a> pandemic hit the very young in addition to the very old. Middle aged adults (ages 24–34) lied somewhere in between.</p>
<p>While COVID-19 does increase the chance of dying in any age group, the baseline for the very young is extremely low to begin with, thereby minimally increasing their overall risk. That cannot be said for older people, who bear the highest risk of getting the severe form of COVID-19 and are therefore also more likely to die.</p>
<p>Demographics play an important role in determining how prevalent severe cases are in any given country. The following distributions could help explain why some countries were or will be hit more severely, even if the rate of infections (percentage of total population infected) is comparable.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/uk-population-2020.png"><img src="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/uk-population-2020.png" alt="Bar chart of United Kingdom's demographics in 2020" /></a></p>
<figcaption>
United Kingdom in 2020
</figcaption>
</figure>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/china-population-2020.png"><img src="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/china-population-2020.png" alt="Bar chart of China's demographics in 2020" /></a></p>
<figcaption>
China in 2020
</figcaption>
</figure>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/japan-population-2020.png"><img src="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/japan-population-2020.png" alt="Bar chart of Japan's demographics in 2020" /></a></p>
<figcaption>
Japan in 2020
</figcaption>
</figure>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/western-europe-population-2020.png"><img src="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/western-europe-population-2020.png" alt="Bar chart of Western Europe's demographics in 2020" /></a></p>
<figcaption>
Western Europe in 2020
</figcaption>
</figure>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/western-europe-population-2020.png"><img src="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/western-europe-population-2020.png" alt="Bar chart of Southern Asia's demographics in 2020" /></a></p>
<figcaption>
Southern Asia in 2020
</figcaption>
</figure>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/western-africa-population-2020.png"><img src="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/western-africa-population-2020.png" alt="Bar chart of Western Africa's demographics in 2020" /></a></p>
<figcaption>
Western Africa in 2020
</figcaption>
</figure>
<h3 id="gender">Gender</h3>
<p>Somewhat surprisingly, COVID-19 affects men more than women, something that's not the case with other viral pneumonias.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/ons-data-2020-03-age-and-gender-covid-19.jpg"><img src="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/ons-data-2020-03-age-and-gender-covid-19.jpg" alt="Graph of age-standardized mortality rate in England and Wales" /></a></p>
<figcaption>
<p>Data from the Office for National Statistics "<a href="https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/deaths/datasets/deathsinvolvingcovid19englandandwalesmarch2020">Deaths involving COVID-19, England and Wales: March 2020</a>" dataset.</p>
</figcaption>
</figure>
<p>The gender difference is especially pronounced when looking at patients requiring intensive care.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/icnarc-age-at-admission-covid-19.png"><img src="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/icnarc-age-at-admission-covid-19.png" alt="Graph of age at admission by sexes" /></a></p>
<figcaption>
<p><a href="https://www.icnarc.org/DataServices/Attachments/Download/c31dd38d-d77b-ea11-9124-00505601089b">ICNARC report on COVID-19 in critical care</a></p>
</figcaption>
</figure>
<h3 id="comorbidities">Comorbidities</h3>
<p>Comorbidities are other diseases that make people more susceptible to COVID-19. In the UK's <a href="https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases">Office for National Statistics data</a>, 91% of cases had at least one of the following comorbidities:</p>
<ul>
<li>Cardiovascular disease</li>
<li>Diabetes</li>
<li>Chronic lung disease</li>
<li>Immunosuppression</li>
<li>Chronic kidney and liver disease</li>
<li>Obesity</li>
</ul>
<p>In addition, some ethnic groups may be more at risk, either due to some comorbidities being more likely within particular groups, their socio-economic status or even genetics.</p>
<h3 id="indirect-deaths">Indirect Deaths</h3>
<p>In addition to direct deaths from the disease given a functional health-care system and optimal care, there are 3 other ways how a pandemic could cause mortalities.</p>
<ul>
<li>Indirect deaths due to health system getting overwhelmed and unable to treat both COVID-19 and other disease patients.<br />
Hasn't occurred in the UK.</li>
<li>Indirect deaths through postponed treatment, screening or elective surgery<sup id="fnref-elective-surgery"><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/#fn-elective-surgery" class="footnote-ref">elective-surgery</a></sup> or general fear of going to the hospital.</li>
<li>Interventions put in place which have a strong economic impact.<br />
Increased deprivation and ill health are strongly correlated.</li>
</ul>
<h3 id="low-income-countries">Low-Income Countries</h3>
<p>For most infections, poverty and malnutrition are correlated with worse outcomes. It's possible mortality rate will be higher in middle and lower income countries. Weaker health systems are more easily overwhelmed. However, occasionally middle and lower income countries have better public health care than expected.</p>
<h2 id="route-of-transmission">Route of Transmission</h2>
<p><strong>Primary routes</strong> of possible transmissions:</p>
<table><thead><tr><th>Method</th><th>Example disease</th></tr></thead><tbody><tr><td>Respiratory</td><td>COVID-19, SARS, MERS, flu</td></tr><tr><td>Sexual or blood</td><td>HIV</td></tr><tr><td>Touch</td><td>Ebola</td></tr><tr><td>Water or food</td><td>Cholera</td></tr><tr><td>Vector (insects, parasites)</td><td>Zika, malaria</td></tr></tbody></table>
<p>Respiratory viruses are most difficult to tackle. They mainly enter via the nose, mouth or possibly eyes. Droplets above 5–10µm come from coughing, sneezing or talking. They're usually carried no more than a meter. Additionally, SARS-CoV-2 could be transmitted via a <strong>secondary route</strong> by touching contaminated surfaces and then your face.</p>
<p>Ways to reduce respiratory transmission would be to catch your cough and washing your hands with soap or alcohol gel. Soap damages the <a href="https://en.wikipedia.org/wiki/Viral_envelope">lipid around the virus</a>. The majority of SARS-CoV-2 will have been gone by 72h on various surfaces, with porous surfaces like cardboard reducing while flat surfaces like plastics or stainless steel prolonging its lifespan.</p>
<p>Some viruses are infectious only when symptoms develop. Unfortunately SARS-CoV-2 seems to be infectious even 2–3 days prior to getting symptoms and infectious even if its asymptomatic. Once people do develop symptoms, it probably takes about a week before the infectiousness drops off significantly. This was investigated in "<a href="https://www.nature.com/articles/s41591-020-0869-5.pdf">Temporal dynamics in viral shedding and transmissibility of COVID-19</a>". Infectiousness even in asymptomatic cases means merely isolating those visibly ill won't be sufficient.</p>
<h3 id="sars"><a href="https://en.wikipedia.org/wiki/Severe_acute_respiratory_syndrome">SARS</a></h3>
<p>The <strong>severe acute respiratory syndrome</strong> has similarities to today's strain, with origins in bats around 2002–2003. Around 8422 were infected at that time, out of which 11% died. Largest outbreak, 63% of total cases, was in China. 20% in Hong Kong. Low hundreds in Taiwan, Canada and Singapore. Very few cases elsewhere. Transmission was probably exclusively from people who had symptoms. Hasn't returned since.</p>
<h3 id="mers"><a href="https://en.wikipedia.org/wiki/Middle_East_respiratory_syndrome">MERS</a></h3>
<p>The <strong>Middle East respiratory syndrome</strong> jumped from <a href="https://en.wikipedia.org/wiki/Dromedary">dromedaries</a>, Arabian camels, to humans around 2012. It too originates from bats, even if camels were the intermediate step. Had a high — 35% — death rate. Had one outbreak in South Korea in 2015 (185 confirmed cases), but didn't take off beyond that.</p>
<h3 id="spanish-flu"><a href="https://en.wikipedia.org/wiki/Spanish_flu">Spanish flu</a></h3>
<p>From 1918–1919, the H1N1 influenza virus caused the death of 17 million up to 100 million people. It did so in 3 waves over 2 years, with the second wave being more deadly than the first.</p>
<p>H1N1 is of the same family that's responsible for the <a href="https://en.wikipedia.org/wiki/2009_swine_flu_pandemic">2009's swine flu</a>, albeit a new strain. The latter strain caused around 43–89 million cases and 8–18 thousand deaths. High transmission, but low mortality.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/h1n1-confirmed-cases.png"><img src="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/h1n1-confirmed-cases.png" alt="Number of confirmed cases of H1N1 in 2009" /></a></p>
<figcaption>
From the World Health Organization
</figcaption>
</figure>
<h3 id="asian-flu"><a href="https://en.wikipedia.org/wiki/Influenza_A_virus_subtype_H2N2">Asian flu</a></h3>
<p>H2N2 resulted in the 1957 Asian Flu pandemic, which had a ~0.6% mortality and killed around 1 to 4 million people worldwide.</p>
<h2 id="force-of-transmission">Force of Transmission</h2>
<p>R<sub>0</sub> or <strong>force of transmission</strong> is how many people one infected person in turn infects. For an epidemic and exponential growth to occur, R<sub>0</sub> has to be above 1. Here's how COVID-19's R<sub>0</sub> in the United Kingdom stacks up against other diseases:</p>
<table><thead><tr><th>Disease</th><th>R<sub>0</sub></th></tr></thead><tbody><tr><td>COVID-19</td><td>3</td></tr><tr><td>Ebola</td><td>1.2–2.5</td></tr><tr><td><a href="https://en.wikipedia.org/wiki/Spanish_flu">Spanish flu from 1918</a></td><td>2–3</td></tr><tr><td>HIV</td><td>2–5 in the expansion phase</td></tr><tr><td>Measles</td><td>>10</td></tr><tr><td>Malaria</td><td>>100</td></tr></tbody></table>
<h3 id="reducing-transmission">Reducing Transmission</h3>
<p>In the absence of a vaccine or drug, relying on social and public health measures could help address the quick growth. One popular theory is "flattening the curve", which is an attempt to reduce the transmission rate through social measures for an extended period of time in hopes of giving the heath-care system time to handle existing cases and possibly expand its capacity.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/flatten-the-curve.jpg"><img src="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/flatten-the-curve.jpg" alt="Exponential infection curve plotted against curve with protected measures" /></a></p>
</figure>
<p>Without vaccines or drugs, there are four approaches:</p>
<ul>
<li>Reduce chance of infection through hygiene.</li>
<li>Isolate cases and close contacts or family.</li>
<li>Reduce contact between households via social distancing.</li>
<li>Reduce chance of infection of the most vulnerable.</li>
</ul>
<p>Its apparently shown that a combination of the measures above are necessary, as none individually are sufficient. There is, however, a risk of R<sub>0</sub> returning to levels above 1 once the measures are lifted. There's a possibility those measures will have to be continued indefinitely until medical countermeasures have been developed <em>or</em> there's evidence a large number of people have been infected asymptomatically.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/second-wave.png"><img src="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/second-wave.png" alt="Graph of cases against months since transmission" /></a></p>
<figcaption>
<p>From "<a href="https://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736(20)30567-5.pdf">How will country-based mitigation measures influence the course of the COVID-19 epidemic?</a>".</p>
</figcaption>
</figure>
<p>The optimal answer on how and when to unwind lockdowns is dependent on whether to focus on direct or indirect deaths. There are important social and economic considerations. Alluding to the flu pandemics earlier, it's possible for the second wave to be more dangerous than the first. That's why it's important to ensure R<sub>0</sub> does not get above 1.</p>
<h3 id="seasonality">Seasonality</h3>
<p>There may be a seasonal element to COVID-19. While winter has always put health-care systems under more pressure due to higher transmissions of respiratory viruses, the current pandemic may exaggerate that. On the other hand, social distancing measures already in place may reduce flu and other upper respiratory tract infections. Observing or identifying COVID based on symptoms may get harder during the winter season as there are plenty of seasonal diseases with similar symptoms. Overall, though, the climate's (temperature, humidity) impact is unknown.</p>
<h2 id="treatments-and-vaccines">Treatments and Vaccines</h2>
<h3 id="vaccines">Vaccines</h3>
<p>Vaccine strategies come in multiple different forms:</p>
<ul>
<li>
<p><strong>Epidemic modifying vaccines</strong> are highly effective at preventing infection. Giving it to the majority of the population means even the few that weren't vaccinated are unlikely to be infected as people around them won't pass the virus on.</p>
</li>
<li>
<p><strong>Disease modifying vaccines</strong> may be only partially effective. They're most useful for high risk groups, as long as they reduce the severity of the disease and complications.</p>
</li>
</ul>
<h3 id="drugs">Drugs</h3>
<p>Many diseases lack vaccines, but can be adequately dealt with by drugs. This could help people with existing mild symptoms from becoming severely ill. Alternatively only severe cases could be treated, avoiding need for intensive care and possible death.</p>
<p>Possible treatments can be grouped into three categories:</p>
<ul>
<li>
<p><strong>Antivirals</strong><br />
Useful early or as prophylaxis. Generally less well developed than antibiotics, although we do have moderately good antivirals for influenza, decent antivirals to suppress HIV or even cure Hepatitis C.</p>
</li>
<li>
<p><strong>Anti-inflammatory drugs</strong> or <strong>immune modulating drugs</strong><br />
There are many feedback loops in the immune system. We've got powerful drugs that can suppress some parts of it, such as steroids, drugs from rheumatology and drugs for organ transplants.</p>
<p>Severe COVID-19 could be from <a href="https://en.wikipedia.org/wiki/Cytokine_release_syndrome">cytokine storm</a>, which is a runaway effect of white blood activating more white blood cells, which then end up causing more damage to the body than the virus itself. If it truly is an auto-immune disease, anti-inflammatory drugs could help. It's possible they could make things worse though.</p>
</li>
<li>
<p><strong>Antibodies</strong><br />
Take the antibodies from recovered people and inject them to patients. This presumes recovered people have antibodies. Eventually we may be able to manufacture antibodies.</p>
</li>
</ul>
<p>While it may seem appealing to begin high-dose treatment given a plausibly effective drug, it's important to do trials before doing so. One study, "<a href="https://jamanetwork.com/journals/jamanetworkopen/articlepdf/2765499/borba_2020_oi_200372.pdf">Effect of High vs Low Doses of Chloroquine Diphosphate as Adjunctive Therapy for Patients Hospitalized With Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) Infection</a>", apparently found that high doses of chloroquine actually caused more deaths than low doses.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/possible-covid-treatments.png"><img src="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/possible-covid-treatments.png" alt="Relation-map of possible COVID-19 treatments" /></a></p>
<figcaption>
Possible COVID-19 treatments and ongoing trials.
</figcaption>
</figure>
<h2 id="unknowns">Unknowns</h2>
<ul>
<li>Proportion of asymptomatic infections.</li>
<li>Immunity duration after recovery</li>
<li>Blood test correlation with immunity</li>
<li>Children's transmission rate.
Relevant for opening schools.</li>
<li>Seasonality</li>
<li>Pathological reason for deterioration after a week.</li>
<li>Gender differences in severe cases.</li>
</ul>
<div class="footnotes">
<hr />
<ol>
<li id="fn-elective-surgery">Elective surgery being non-emergency surgeries, but still done to preserve health.<a href="https://m811.com/articles/notes-on-covid-19-an-initial-view-talk/#fnref-elective-surgery" class="footnote-backref">↩</a></li>
</ol>
</div>https://m811.com/articles/notes-on-living-with-1000-open-source-projects-talk/Notes on «Living with 1000 Open Source Projects» talk2020-03-05T01:15:30.000Z2020-03-05T01:15:30.000Z<big>
<p>A short and a funny talk about how to manage a large number of open source projects in a sustainable way. Occasionally tongue in cheek and not taking itself too seriously. A refreshing viewpoint compared to the 2020s open source sustainability philosophy of presuming every open source project deserves a future and its maintainers a payment.</p>
</big>
<h2 id="open-source-developers">Open Source Developers</h2>
<p>Nic thinks there are two types of open source developers and contributors.</p>
<ol>
<li>Type A are enthusiastic developers. They announce their projects, help their users both use the project and occasionally even learn the language. They write blog posts and ask others to write blog posts. They share those blog posts and so on.</li>
<li>Second, there are those that used to be of type A.</li>
</ol>
<p>Some people give up silently, avoiding their projects and maybe the Internet as a whole. A few do it publicly. <a href="https://weblog.jamisbuck.org">Jamis Buck</a>, for example, quit <a href="https://capistranorb.com">Capistrano</a> and its related networking libraries back in 2009 in his <a href="https://weblog.jamisbuck.org/2009/2/24/net-ssh-capistrano-and-saying-goodbye.html">"Net::SSH, Capistrano, and Saying Goodbye"</a> post. Ironically, I've been <em>silently</em> ignoring maintaining a <a href="https://github.com/moll/capistrano-rsync">Capistrano plugin</a> of mine for about 7 years now.</p>
<p>While Nic says he doesn't have projects as popular as Jamis', he has ended up with numerable smaller repositories over the past years. He also thinks there are probably more people with crappy projects than with popular projects like Capistrano that need talking to.</p>
<p>Justifiably, with only 168h per week, he finds it impossible to tend to each of his projects. It's necessary to reduce the amount of time spent on any project. He illustrates this with an analogy of pets.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-living-with-1000-open-source-projects-talk/slide13.jpg"><img src="https://m811.com/articles/notes-on-living-with-1000-open-source-projects-talk/slide13.jpg" alt="Cute pet dog vs a pack of dogs" /></a></p>
</figure>
<p>Like children or pets, pet projects don't scale well. A few are fun, but more than a handful can't work. First, Nic suggests you pick a few to be ones you'll care deeply about. For those you may end being known for. Second, projects you perhaps needed for work and happened to publish as an open source library, should be designated as work. That is, their maintenance is something you'll also be doing during work-hours. That "scales as much as work does". The rest you should just ditch.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-living-with-1000-open-source-projects-talk/slide17.jpg"><img src="https://m811.com/articles/notes-on-living-with-1000-open-source-projects-talk/slide17.jpg" alt="David Heinemeier Hansson's MIT license summary "I Don't Owe You Shit"" /></a></p>
</figure>
<h2 id="reducing-your-maintenance-burden">Reducing Your Maintenance Burden</h2>
<p>The next goal is driving down the maintenance effort. As a result, you may occasionally see people complaining about your project's status online. Nic finds it unfair. Just because you open sourced it doesn't imply you've bear the obligation for maintaining it. He does, however, have a few ideas how to minimize the impact of reduced attention and manage expectations.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-living-with-1000-open-source-projects-talk/slide20.jpg"><img src="https://m811.com/articles/notes-on-living-with-1000-open-source-projects-talk/slide20.jpg" alt="Reduce bad karma" /></a></p>
</figure>
<p>You could publish the project status on its page. The number of days since the last commit could be a good indicator. For handling questions and assistance requests, you could set up a mailing list or forum where people could come help each other. These days GitHub Issues would probably suffice, if you're hosting your code there.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-living-with-1000-open-source-projects-talk/slide22.jpg"><img src="https://m811.com/articles/notes-on-living-with-1000-open-source-projects-talk/slide22.jpg" alt="GitHub project forks graph" /></a></p>
</figure>
<p>In addition to solving communication, helping people find each other's patches could reduce your workload. GitHub's project fork view will in come in handy. People could then pick a more up to date fork or collect patches themselves.</p>
<p>Instead of still considering the abandoned projects as <em>your</em> projects, Nic suggests viewing them as ones you're merely looking over. This could permit you to share commit rights more freely.</p>
<p>I'm not sure I personally agree with that liberal view of commit access. Your original dependents may have chosen your project not only for its merit, but for your reputation. Giving maintenance permission to a seemingly helpful yet random contributor would betray that trust <em>and</em> possibly cause a security nightmare for your users. Unfortunately, we're not talking about a hypotheticals either. A recent and popular case showed up in late 2018 where someone taking over the <code class="">event-stream</code> Node.js module from its original maintainer ended up back-dooring it. This was well described in <a href="https://snyk.io/blog/a-post-mortem-of-the-malicious-event-stream-backdoor/">Snyk's post-mortem of event-stream</a>.</p>
<h2 id="driving-down-maintenance-cost">Driving Down Maintenance Cost</h2>
<p>Nic mentions libraries of Rake tasks (like <a href="https://rubygems.org/gems/hoe">Hoe</a>) to package and deploy projects. Having a Rakefile at hand means less thinking of the process, further saving you time and reducing the need to help others manually.</p>
<p>Reviewing bug reports could take up to half an hour to reproduce the bug. To avoid having to deal with bug reports in the first place, Nic suggests taking quality control more seriously and writing tests. He's using <a href="https://github.com/seattlerb/zentest">MultiRuby from ZenTest</a> to run tests against different Ruby versions.</p>
<h2 id="links">Links</h2>
<ul>
<li><a href="https://github.com/drnic">Nic Williams' GitHub profile</a></li>
</ul>https://m811.com/articles/notes-on-new-features-in-c-talk/Notes on «"New" Features in C» talk2019-10-01T12:01:52.000Z2019-10-01T12:01:52.000Z<big>
A non-exhaustive overview of what has changed since 1989 when ANSI approved C89.
</big>
<h2 id="c-standards">C Standards</h2>
<table><thead><tr><th>Year</th><th>Event</th></tr></thead><tbody><tr><td>1983</td><td>C Standards commitee created.</td></tr><tr><td>1989</td><td>ANSI C89, the first USA Standard.</td></tr><tr><td>1990</td><td>ISO C90, the first international standard.<br />Identical to the ANSI version. "C90" is often used to refer to both C89 and C90.</td></tr><tr><td>1999</td><td><a href="https://en.wikipedia.org/wiki/C99">ISO C99</a> (ISO/IEC 9899:1999).</td></tr><tr><td>2011</td><td><a href="https://en.wikipedia.org/wiki/C11_(C_standard_revision)">ISO C11</a> (ISO/IEC 9899:2011)</td></tr><tr><td>2018</td><td><a href="https://en.wikipedia.org/wiki/C18_(C_standard_revision)">ISO C18</a> (ISO/IEC 9899:2018).<br />Said to be identical to C11 except wording.</td></tr></tbody></table>
<p>The latest <strong>publicly available</strong> C standard from 2007 is <a href="http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf">ISO/IEC 9899:TC3</a>, available from the <a href="http://www.open-std.org/jtc1/sc22/wg14/www/standards">C working group site</a>. The latest ISO specification, <a href="https://www.iso.org/standard/74528.html">ISO/IEC 9899:2018</a>, however is behind ISO's paywall for €181. I think it's a <strong>disgrace for such a widely used programming language to have its official specification paywalled</strong>, and rather ironic given how much of the world's open-source software is in C. The <a href="https://en.wikipedia.org/wiki/GNU_Project">GNU project</a> gave people a free compiler, <a href="https://gcc.gnu.org">GCC</a>, 32 years ago. Perhaps it's time to give people free C, too. Fortunately the C working group at one point distributed the <a href="https://web.archive.org/web/20181230041359if_/http://www.open-std.org/jtc1/sc22/wg14/www/abq/c17_updated_proposed_fdis.pdf">working draft for ISO/IEC 9899:2017</a>. Thanks to Archive.org, it's still available. It just doesn't seem to be "officially available", nor is it the actual final standard.</p>
<p>I see similarities between paywalling standards and paywalling scientific literature. The latter practice is fortunately starting to change, thanks to civil disobedience and the technical help of <a href="https://en.wikipedia.org/wiki/Sci-Hub">Sci-Hub</a> for providing people access to research they (even if partly) funded through tax. I'm imagining something like this ought to occur in the standards world, too. There's little ethical justification for locking up standards that are based partly on publicly funded work, or worse yet, referred to in laws.</p>
<h3 id="identifying-the-standard-version">Identifying the Standard Version</h3>
<p>C90 introduced the <code class="">__STDC__</code> macro to identify a standard compliant C90 compiler. However, as <code class="">__STDC__</code> was only defined to return <code class="">1</code>, it couldn't later be used to distinguish between C90 and C99. For that, <code class="">__STDC_VERSION__</code> was added. <code class="">__STDC_VERSION__</code> was defined to return an integer representing the year and month of the standard — <code class="">201710</code> for C18, for example.</p>
<h2 id="identifiers">Identifiers</h2>
<p>The C standard is said to consider not breaking existing code as far more important than breaking existing implementations. Newer standards and introduced keywords therefore may not conflict with existing code.</p>
<p><code class=""><stdio.h></code> declares a series of public identifiers:</p>
<p><code class="">EOF</code> | Macro
<code class="">FILE</code> | Type
<code class="">printf</code> | Function
<code class="">stdout</code> | "object"</p>
<p>Behind the scenes, they're implemented in terms of undocumented identifiers. As an example, <code class="">FILE</code> in GCC v9 is implemented in terms of a structure named <code class="">_IO_FILE</code>:</p>
<pre><code class="hljs language-c"><span class="hljs-class"><span class="hljs-keyword">struct</span> _<span class="hljs-title">IO_FILE</span>;</span>
<span class="hljs-keyword">typedef</span> <span class="hljs-class"><span class="hljs-keyword">struct</span> _<span class="hljs-title">IO_FILE</span> <span class="hljs-title">FILE</span>;</span>
</code></pre>
<p>Another source of undocumented identifiers are header guards, such as those used to implement the standard's requirement that any standard header may be included (e.g. <code class="">#include <stdio.h></code>) more than once. They way these are often implemented is through a preprocessor variable at the top of the header.</p>
<pre><code class="hljs language-c"><span class="hljs-meta">#<span class="hljs-meta-keyword">ifndef</span> _STDIO_H</span>
<span class="hljs-meta">#<span class="hljs-meta-keyword">define</span> _STDIO_H 1</span>
<span class="hljs-comment">// ...</span>
<span class="hljs-meta">#<span class="hljs-meta-keyword">endif</span></span>
</code></pre>
<p>To ensure the structure behind <code class="">FILE</code>, <code class="">_iobuf</code>, or the header-guard, <code class="">_STDIO_H</code>, don't conflict with people's code, the C standard has a concept of reserved identifiers.</p>
<ol>
<li>
<p>Keywords are forbidden everywhere.<br />
These would be, among a few others:</p>
<ul class="inline">
<li><code class="">auto</code></li>
<li><code class="">break</code></li>
<li><code class="">case</code></li>
<li><code class="">char</code></li>
<li><code class="">const</code></li>
<li><code class="">continue</code></li>
<li><code class="">default</code></li>
<li><code class="">do</code></li>
<li><code class="">double</code></li>
<li><code class="">else</code></li>
<li><code class="">enum</code></li>
<li><code class="">extern</code></li>
<li><code class="">float</code></li>
<li><code class="">for</code></li>
<li><code class="">goto</code></li>
<li><code class="">if</code></li>
<li><code class="">inline</code></li>
<li><code class="">int</code></li>
<li><code class="">long</code></li>
<li><code class="">register</code></li>
<li><code class="">restrict</code></li>
<li><code class="">return</code></li>
<li><code class="">short</code></li>
<li><code class="">signed</code></li>
<li><code class="">sizeof</code></li>
<li><code class="">static</code></li>
<li><code class="">struct</code></li>
<li><code class="">switch</code></li>
<li><code class="">typedef</code></li>
<li><code class="">union</code></li>
<li><code class="">unsigned</code></li>
<li><code class="">void</code></li>
<li><code class="">volatile</code></li>
<li><code class="">while</code></li>
</ul>
</li>
<li>
<p>Identifiers beginning with two underscores are reserved everywhere.</p>
<pre><code class="hljs language-c"><span class="hljs-class"><span class="hljs-keyword">struct</span> <span class="hljs-title">foo</span> {</span> <span class="hljs-keyword">double</span> __bar; };
</code></pre>
</li>
<li>
<p>Identifiers beginning with a single underscore and a lowercase character are reserved only at the global scope.<br />
They are therefore permitted to be used in structures or variables:</p>
<pre><code class="hljs language-c"><span class="hljs-class"><span class="hljs-keyword">struct</span> <span class="hljs-title">foo</span> {</span> <span class="hljs-keyword">double</span> _bar; };
<span class="hljs-function"><span class="hljs-keyword">int</span> <span class="hljs-title">main</span><span class="hljs-params">()</span> </span>{ <span class="hljs-keyword">int</span> _foo; }
</code></pre>
</li>
<li>
<p>Identifiers beginning with an underscore and an uppercase character are again reserved everywhere.</p>
<pre><code class="hljs language-c"><span class="hljs-class"><span class="hljs-keyword">struct</span> <span class="hljs-title">foo</span> {</span> <span class="hljs-keyword">double</span> _Bar; };
</code></pre>
</li>
</ol>
<p>Here's how this played out in C99, which introduced the boolean type. C99 couldn't break existing code, therefore couldn't also introduce the <code class="">bool</code> keyword like C++ had. They could only use one of the reserved naming conventions above. They ended up adding <code class="">_Bool</code> as the type, while adding the familiar <code class="">bool</code> and <code class="">true</code>/<code class="">false</code> words as macros in a separate file, <code class=""><stdbool.h></code>:</p>
<pre><code class="hljs language-c"><span class="hljs-meta">#<span class="hljs-meta-keyword">define</span> bool _Bool</span>
<span class="hljs-meta">#<span class="hljs-meta-keyword">define</span> true 1</span>
<span class="hljs-meta">#<span class="hljs-meta-keyword">define</span> false 0</span>
</code></pre>
<p>As defining them out of the box could interfere with existing code, <code class=""><stdbool.h></code> needs to be included explicitly.</p>
<p>Dan explains that C++ needed a separate boolean type immediately for its overloading. A mere type definition or type alias wouldn't have been different in the eyes of the compiler for overloading to work.</p>
<h2 id="c99-features">C99 Features</h2>
<p>C99 added <code class="">//</code> comments.</p>
<h3 id="integer-types">Integer Types</h3>
<p>C has implementation-defined sizes for integer types, which could be an annoyance to people that need to work with specific sizes. For a long time people were working around that with compiler and system specific type definitions, such as:</p>
<pre><code class="hljs language-c"><span class="hljs-keyword">typedef</span> <span class="hljs-keyword">char</span> int8;
<span class="hljs-keyword">typedef</span> <span class="hljs-keyword">unsigned</span> <span class="hljs-keyword">long</span> uint32;
</code></pre>
<p>C99 finally standardized these and defined them in <code class=""><stdint.h></code>. They take the form of <code class="">int8_t</code>, <code class="">int16_t</code>, <code class="">uint32_t</code> and so on to cover all 8–64bit variants. Note however that the standard doesn't require that you get all of the integer sizes, only that those you get follow the pattern above. As C was designed for a wide variety of architectures, some of those may not have small word sizes. By definition a byte in C is the smallest addressable unit. Dan mentioned digital signal processors that could have 32-bit word-sizes.</p>
<p>Aside from exact-size integers, C99 also defined minimum-size integers: <code class="">int_least16_t</code> and <code class="">uint_least16_t</code> et alii. Be careful as arithmetic, when depending on two's-complement or overflow aspects, may not work as expected because the compiler is free to choose larger integers.</p>
<p>There's also a fast-integer concept with <code class="">int_fast16_t</code> and others, which you can use when you need N-bits to do arithmetic and prefer to get a size where that is the fastest. As with minimum-size integers, some architectures may choose larger sizes, like ARM, where 32-bit arithmetic is faster than 16-bit or 8-bit because smaller sizes require conversion penalties. Fast-integer could be useful for loop iterator variables.</p>
<p>Finally there's a greatest-size integer type <code class="">intmax_t</code> and <code class="">uintmax_t</code>.</p>
<p>Somewhat related, an integer types capable of containing pointers to objects (not to functions) were also introduced: <code class="">intptr_t</code> and <code class="">uintptr_t</code>. Where regular pointers in C only support addition and subtraction, these support multiplication and division. They are, however, optional, as there could be systems where pointers are not representable as integers. Ivan Godard's Mill CPU would be an example. Ivan talked about the <a href="https://www.youtube.com/watch?v=QyzlEYspqTI">architecture of the Mill CPU</a> at an LLVM meetup in 2015.</p>
<h3 id="long-long-integers">Long Long Integers</h3>
<p>In addition to the fixed-size integer types, C99 added <code class="">long long</code>. It wasn't just a task to add the type, the supporting library functions required additions too. Namely, <code class="">atoll</code>, for "ascii to long long" and <code class="">llabs</code> for getting the absolute value of a <code class="">long long</code>. <code class="">printf</code> also needed a <code class="">%lld</code> format string.</p>
<h3 id="complex-numbers">Complex Numbers</h3>
<p>C99 also added support for both complex numbers (<code class="">x + yi</code>) with <code class="">_Complex</code> and imaginary numbers with <code class="">_Imaginary</code>. As before, nicer names are behind a header file:</p>
<pre><code class="hljs language-c"><span class="hljs-meta">#<span class="hljs-meta-keyword">include</span> <span class="hljs-meta-string"><complex.h></span></span>
<span class="hljs-keyword">double</span> <span class="hljs-built_in">complex</span> foo = <span class="hljs-number">13</span> + <span class="hljs-number">37</span> * I;
</code></pre>
<p>C does implicit conversions between regular integers and complex integers by either assuming the imaginary part is zero or by discarding the imaginary part. There are also <code class="">creal</code> and <code class="">cimag</code> functions to get the real and imaginary parts.</p>
<p>Complex number types are mandatory for compilers, but the lone `imaginary part is optional.</p>
<h3 id="function-name">Function Name</h3>
<p>C99 introduced <code class="">__func__</code> that gets you the name of the local function for debugging and function call tracing. It's use is likely to get you a statically allocated null-terminated string:</p>
<pre><code class="hljs language-c"><span class="hljs-function"><span class="hljs-keyword">void</span> <span class="hljs-title">foo</span><span class="hljs-params">()</span> </span>{
__func__; <span class="hljs-comment">// => static char const __func__[] = "foo";</span>
}
</code></pre>
<h3 id="declaration-ordering">Declaration Ordering</h3>
<p>C90 didn't allow declarations after the first statement in a block, i.e a function call. It required all definitions to happen at the top, even if initialization (first assignment) happened a lot later. C99 lets you mix declarations and statements:</p>
<pre><code class="hljs language-c"><span class="hljs-function"><span class="hljs-keyword">void</span> <span class="hljs-title">foo</span><span class="hljs-params">()</span> </span>{
<span class="hljs-keyword">int</span> foo;
bar();
<span class="hljs-keyword">int</span> baz;
}
</code></pre>
<p>This aligns with good programming practice on reducing the scope of used variables by permitting their declaration to happen closer to their place of use.</p>
<p>It was natural to also permit delayed declaration for loop variables, so it was no longer necessary to define <code class="">i</code> at the top of the function with a loop inside:</p>
<pre><code class="hljs language-c"><span class="hljs-keyword">for</span> (<span class="hljs-keyword">int</span> i = <span class="hljs-number">0</span>; i < n; ++i)
<span class="hljs-comment">// ....</span>
}
</code></pre>
<p>C99 also scopes <code class="">i</code> in the above example to within the loop. A later loop can reuse the same name.</p>
<h3 id="inline-functions">Inline Functions</h3>
<p>C99 also introduced the <code class="">inline</code> keyword for functions, permitting people to migrate from error-prone macros to semantically true functions with none of the overhead of function calls.</p>
<pre><code class="hljs language-c"><span class="hljs-function"><span class="hljs-keyword">inline</span> <span class="hljs-keyword">int</span> <span class="hljs-title">max</span><span class="hljs-params">(<span class="hljs-keyword">int</span> a, <span class="hljs-keyword">int</span> b)</span> </span>{ <span class="hljs-keyword">return</span> a > b ? a : b; }
</code></pre>
<p>Another positive of native inline functions is that you can take the address of the function if necessary.</p>
<p>Inline functions have to be defined, not only declared, in an header file for the compiler to be able to inline their contents. However that raises a problem — the inline function could end up being defined in multiple object files, such as when you take the function's address, and the linker may complain that it's seeing duplicates. Whereas C++ decided to have the linker ignore identical duplicates, C has you pick a place for the implementation to go if you need to refer to its non-inline variant. You do that by declaring, but not defining, the function in a file of your choice.</p>
<pre><code class="hljs language-c"><span class="hljs-meta">#<span class="hljs-meta-keyword">include</span> <span class="hljs-meta-string">"max.h"</span></span>
<span class="hljs-function"><span class="hljs-keyword">int</span> <span class="hljs-title">max</span><span class="hljs-params">(<span class="hljs-keyword">int</span>, <span class="hljs-keyword">int</span>)</span></span>;
</code></pre>
<p>You could prefix the declaration above with <code class="">extern</code> or <code class="">extern inline</code>, but not with <code class="">inline</code>.</p>
<p>Dan mentions that GCC may support C++-like inline function semantics when compiling C by requiring you to pick a place for the inline function. Depending on those features will then no longer be standard C.</p>
<h3 id="compound-literals">Compound Literals</h3>
<p>If you made a rational number type and want to use it <em>inline</em>, C90 requires you to first declarate it as a variable, initialize it and only then can you use it:</p>
<pre><code class="hljs language-c"><span class="hljs-class"><span class="hljs-keyword">struct</span> <span class="hljs-title">rational</span> {</span> <span class="hljs-keyword">long</span> numerator, denominator; };
<span class="hljs-class"><span class="hljs-keyword">struct</span> <span class="hljs-title">rational</span> <span class="hljs-title">half</span> =</span> {<span class="hljs-number">1</span>, <span class="hljs-number">2</span>};
<span class="hljs-class"><span class="hljs-keyword">struct</span> <span class="hljs-title">rational</span> <span class="hljs-title">another_half</span> =</span> {<span class="hljs-number">2</span>, <span class="hljs-number">4</span>};
is_equal(half, another_half);
</code></pre>
<p>C99 introduces compound literals, where you can "cast" a compound literal to your structure:</p>
<pre><code class="hljs language-c">is_equal(half, (rational) {<span class="hljs-number">2</span>, <span class="hljs-number">4</span>});
</code></pre>
<p>This also works for arrays:</p>
<pre><code class="hljs language-c"><span class="hljs-built_in">memcmp</span>(some_array, (<span class="hljs-keyword">int</span>[]) {<span class="hljs-number">1</span>, <span class="hljs-number">2</span>, <span class="hljs-number">3</span>, <span class="hljs-number">4</span>}, <span class="hljs-number">4</span>);
</code></pre>
<p>Relatedly, C90 only permitted initializing the first member of the union:</p>
<pre><code class="hljs language-c"><span class="hljs-class"><span class="hljs-keyword">union</span> <span class="hljs-title">floaty</span> {</span> <span class="hljs-keyword">int</span> i; <span class="hljs-keyword">float</span> f; };
<span class="hljs-class"><span class="hljs-keyword">union</span> <span class="hljs-title">floaty</span> <span class="hljs-title">n</span> =</span> {<span class="hljs-number">42</span>};
<span class="hljs-class"><span class="hljs-keyword">union</span> <span class="hljs-title">floaty</span> <span class="hljs-title">n</span> =</span> {<span class="hljs-number">13.37</span>};
</code></pre>
<p>Even if you assigned a float value like in the last line, at best you got a conversion warning, but it still initialized the integer. C99 introduced <strong>designated initializers</strong>, which let you specify which member of the union you're initializing:</p>
<pre><code class="hljs language-c"><span class="hljs-class"><span class="hljs-keyword">union</span> <span class="hljs-title">floaty</span> <span class="hljs-title">n</span> =</span> {.i = <span class="hljs-number">42</span>};
<span class="hljs-class"><span class="hljs-keyword">union</span> <span class="hljs-title">floaty</span> <span class="hljs-title">n</span> =</span> {.f = <span class="hljs-number">13.37</span>};
</code></pre>
<p>This also works when initializing regular structure members:</p>
<pre><code class="hljs language-c"><span class="hljs-class"><span class="hljs-keyword">struct</span> <span class="hljs-title">date</span> {</span> <span class="hljs-keyword">int</span> year; <span class="hljs-keyword">int</span> month; <span class="hljs-keyword">int</span> day; };
<span class="hljs-class"><span class="hljs-keyword">struct</span> <span class="hljs-title">date</span> <span class="hljs-title">today</span> =</span> {.year = <span class="hljs-number">2015</span>, .month = <span class="hljs-number">6</span>, .day = <span class="hljs-number">18</span>};
</code></pre>
<p>And for arrays, which lets you specify which of the elements are non-zero:</p>
<pre><code class="hljs language-c"><span class="hljs-class"><span class="hljs-keyword">struct</span> <span class="hljs-title">date</span> {</span> <span class="hljs-keyword">int</span> year; <span class="hljs-keyword">int</span> month; <span class="hljs-keyword">int</span> day; };
<span class="hljs-keyword">int</span> numbers[<span class="hljs-number">10</span>] = {[<span class="hljs-number">1</span>] = <span class="hljs-number">10</span>, [<span class="hljs-number">5</span>] = <span class="hljs-number">20</span>};
</code></pre>
<h3 id="variable-length-arrays">Variable-Length Arrays</h3>
<p>In C90 there's no way to define an array with a dynamic number of elements. That becomes a problem when you've got a function that takes the number of elements and needs to allocate it. The only option was manual memory allocation and later freeing. That was especially error-prone when multi-dimension arrays were used:</p>
<pre><code class="hljs language-c"><span class="hljs-function"><span class="hljs-keyword">void</span> <span class="hljs-title">iterate</span><span class="hljs-params">(<span class="hljs-keyword">size_t</span> rows, <span class="hljs-keyword">size_t</span> columns)</span> </span>{
<span class="hljs-keyword">int</span>* matrix = <span class="hljs-built_in">malloc</span>(rows * columns * <span class="hljs-keyword">sizeof</span>(<span class="hljs-keyword">int</span>));
<span class="hljs-comment">// ...</span>
<span class="hljs-built_in">free</span>(matrix);
}
</code></pre>
<p>You couldn't use multiple brackets to refer to nested elements (<code class="">matrix[1][2]</code>), and had to do pointer arithmetic — <code class="">matrix[columns * i + j]</code>.</p>
<p>C99 introduced variable-length arrays with non-constant dimensions, that are both cleaned up automatically and support regular array dereferencing:</p>
<pre><code class="hljs language-c"><span class="hljs-function"><span class="hljs-keyword">void</span> <span class="hljs-title">iterate</span><span class="hljs-params">(<span class="hljs-keyword">size_t</span> rows, <span class="hljs-keyword">size_t</span> columns)</span> </span>{
<span class="hljs-keyword">int</span>* matrix[rows][columns];
<span class="hljs-comment">// ...</span>
}
</code></pre>
<p>Variable-length arrays also expand the definitions of <code class="">sizeof</code> to become a run-time computation when the array dimensions are not known statically.</p>
<p>As only GCC added support for VLAs, the standard committee reclassified them as <strong>optional for C11</strong>.</p>
<p>VLAs also carried over to be used within structures, permitting arrays of dynamic size:</p>
<pre><code class="hljs language-c"><span class="hljs-class"><span class="hljs-keyword">struct</span> <span class="hljs-title">packet</span> {</span>
<span class="hljs-keyword">char</span> header[<span class="hljs-number">10</span>];
<span class="hljs-keyword">char</span> data[];
};
</code></pre>
<p>Allocating space with <code class="">malloc(sizeof(packet) + n * sizeof(char))</code> would then do the right thing — take into account the header and possible padding before the dynamically sized data array.</p>
<h3 id="restricted-pointers">Restricted Pointers</h3>
<p>The possibility of aliased pointers, that is two or more pointers pointing to the same data, preclude a number of optimizations, such as reordering memory accesses or the use of vector instructions on supported CPUs. C99 added a <code class="">restrict</code> qualifier for pointers to hint that there are no other pointers pointing to the same data.</p>
<p>C99's <code class="">memcpy</code> was redefined with <code class="">restrict</code> qualifiers:</p>
<pre><code class="hljs language-c"><span class="hljs-function"><span class="hljs-keyword">void</span>* <span class="hljs-title">memcpy</span><span class="hljs-params">(<span class="hljs-keyword">void</span>* <span class="hljs-keyword">restrict</span> dest, <span class="hljs-keyword">const</span> <span class="hljs-keyword">void</span>* <span class="hljs-keyword">restrict</span> source, <span class="hljs-keyword">size_t</span> count)</span></span>;
</code></pre>
<p><code class="">memcpy</code> therefore expects neither the source nor destination overlap. If they do turn out to overlap, behavior is undefined. <code class="">memmove</code>, on the other hand, is defined to work for overlapping ranges and does not declare its pointer arguments to be restricted:</p>
<pre><code class="hljs language-c"><span class="hljs-function"><span class="hljs-keyword">void</span>* <span class="hljs-title">memmove</span><span class="hljs-params">(<span class="hljs-keyword">void</span>* dest, <span class="hljs-keyword">const</span> <span class="hljs-keyword">void</span>* source, <span class="hljs-keyword">size_t</span> count)</span></span>;
</code></pre>https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/Notes on «The Three Terrible Ideas Weakening Gen Z and Damaging Universities and Democracies» talk2019-09-24T17:44:18.000Z2019-09-24T17:44:18.000Z<big>
<p>Jonathan Haidt cherry-picks a few problems affecting America's newest generation from his and Greg Lukianoff's book "<a href="https://www.thecoddling.com">The Coddling of the American Mind</a>". He draws evidence from two negative trends: the decline of mental health, as measured by instances of depression and self-harm, and political polarization. Both could partly be explained by changes in parenting from mid-90s, primarily overprotecting kids, leading to fragile personalities with poor social skills.</p>
</big>
<h2 id="mental-health">Mental Health</h2>
<p>It's important to distinguish between the previous generation, millennials born 1982–1998 and Gen Z, born after 1995. The two have had different childhoods, and it's only among the latter that mental problems are on the rise.</p>
<p>First, Haidt brought up differences in experiences between the generations at the age of 17–18 (12th graders).</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/12th-graders-experiences.jpg"><img src="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/12th-graders-experiences.jpg" alt="Graph showing 12th grader experiences" /></a></p>
<figcaption>
<p>12th grader experiences with driving, alcohol, dating and working from <a href="http://www.jeantwenge.com/igen-book-by-dr-jean-twenge/">iGen</a> by Jean Twenge.</p>
</figcaption>
</figure>
<p>Instead of driving, drinking, dating or working, Gen Z boys fill their time with video games and girls with social media.</p>
<p>Looking at rates of major depression diagnosed with symptom checklists, there's a rise from around 2010, especially among girls.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/percentage-of-us-teens-with-depression.jpg"><img src="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/percentage-of-us-teens-with-depression.jpg" alt="% of US teens who had a major depression in the last year" /></a></p>
</figure>
<p>Before 2010, things were consistent, with girls having higher rates of depression and boys higher rates of violence and alcoholism. From 2010s onward, when the student body consisted only of Gen Z students, the percentage of people who self-identified as having a psychological disorder like depression, rose steadily, from around 6% to 14% for women and from 2% to 6% for men. Haidt believes computers could be involved as Gen Z got their devices in middle school while millennials, the group with fewer problems, didn't get theirs until college or later.</p>
<p>Counselling center rates seem to confirm the rise, with only anxiety and depression being affected. Other issues like stress are unaffected, so it's less likely Gen Z is merely more open to talk about mental issues. Haidt claims it's rather that they've never learned how to deal with normal everyday problems.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/counceling-concerns-from-2013-to-2018.jpg"><img src="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/counceling-concerns-from-2013-to-2018.jpg" alt="" /></a></p>
</figure>
<p>Jonathan then brings up an article from The New York Times, <a href="https://www.nytimes.com/2018/09/07/opinion/sunday/teenager-anxiety-phones-social-media.html">"The Big Myth About Teenage Anxiety"</a> by Richard A. Friedman, that's critical of Haidt's claims. Friedman says the evidence for the rise in depression is purely based on self-reporting. Haidt believes Friedman is wrong because of the objective empirical evidence. For support, Haidt brings up "<a href="https://jamanetwork.com/journals/jama/fullarticle/2664031">Trends in Emergency Department Visits for Nonfatal Self-inflicted Injuries Among Youth Aged 10 to 24 Years in the United States, 2001-2015</a>". From 2001 until 2015, the number of boys aged 15–24 that have harmed themselves enough to require hospitalization remained consistent, hovering around 240 per 100 000 people a year.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/hospital-admissions-for-non-fatal-self-harm-boys.png"><img src="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/hospital-admissions-for-non-fatal-self-harm-boys.png" alt="" /></a></p>
</figure>
<p>Self-harm among teenage Gen Z girls, on the other hand, increased in 2009, moving from around 350 to 600 per 100 000 a year. Millennials, who were around 20–24 at that time, were unaffected.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/hospital-admissions-for-non-fatal-self-harm-girls.png"><img src="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/hospital-admissions-for-non-fatal-self-harm-girls.png" alt="" /></a></p>
</figure>
<p>Jonathan's conclusion from the data is that getting social media in middle school could be driving the increase in depression and self-harm. He draws attention to the fact that the youngest group of girls, 10–14, didn't use to hurt themselves before 2009. This same is happening with suicide rates.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/suicide-per-100000-by-sex-ages-15-to-19.jpg"><img src="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/suicide-per-100000-by-sex-ages-15-to-19.jpg" alt="" /></a></p>
</figure>
<p>While suicide is going down around the world, it's rising in the US. Male suicide rates are up 25% from the turn of the century (equally so in both young and older men). The rise for girls aged 15–19 is up 70% and for 10–14 girls, whose baseline was otherwise very low, it's up 151%.</p>
<p>Business people are said to notice the trend of anxiety in campuses and among new Gen Z employees, with some executives joking about hearing calls from mothers of the employees.</p>
<h2 id="political-polarization">Political Polarization</h2>
<p>One way to measure polarization is to look at how easy it is to predict votes given the position of the person on the left-right spectrum. If everybody's voting strictly within party lines, it's fair to say people are polarized. That was the case in 1879–1930. For variety of reasons polarization plummeted around 1930–1960s, but then racked up to even higher levels in the 1980–2000s.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/distance-between-the-parties-1879-to-2015.png"><img src="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/distance-between-the-parties-1879-to-2015.png" alt="Polarization of congressional parties graph" /></a></p>
<figcaption>
<p>Graph from <a href="https://legacy.voteview.com/political_polarization_2015.htm">Voteview</a>.</p>
</figcaption>
</figure>
<p>The Pew Research Center has looked into the <a href="https://www.people-press.org/2017/10/05/the-partisan-divide-on-political-values-grows-even-wider/">divide on political values</a> (<a href="https://www.people-press.org/wp-content/uploads/sites/4/2017/10/10-05-2017-Political-landscape-release-updt..pdf">complete report in PDF</a>) through a questionnaire with 10 items, and has divided that up by demographic. That is, can you tell something about people's answers based on their gender, education, race or political party.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/pew-2017-partisan-divide.png"><img src="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/pew-2017-partisan-divide.png" alt="Graph on partisan divides over political values" /></a></p>
</figure>
<p>If you look at the absolute difference by gender, for example, you can see it's been relatively stable throughout the two decades. Differences between religious groups has doubled from 5–6 to 11 points. Neither is as drastic as the gap with regards to political party affiliation, where the difference has climbed from the 2000s at 15 points to 36 points by around 2017.</p>
<h3 id="ten-reasons-politics-are-worse-than-ever">Ten Reasons Politics Are Worse Than Ever</h3>
<p>Ten reasons Haidt posited <a href="https://www.washingtonpost.com/news/wonk/wp/2015/01/07/the-top-10-reasons-american-politics-are-worse-than-ever/">ten reasons</a> along with political scientist Sam Abrams in Washington Post. He didn't elaborate in this talk, but just listed them:</p>
<ol>
<li>Party realignment and purification (1964–1992)</li>
<li>Mass sorting of liberal versus conservative voters (–1990)</li>
<li>Changes in Congress (1995–death of friendships)</li>
<li>Media fractionation (1980s) and the Internet (1990s)</li>
<li>Residential homogeneity and urban versus rural (1990s)</li>
<li>End of cold war and loss of common enemy (1990s)</li>
<li>Increasing immigrations and racial diversity (1990s)</li>
<li>Increasing role of money and negative advertising (2000s)</li>
<li>Generational changing of the guard (1990s)</li>
<li>Increasing education (1970s–)</li>
</ol>
<p>He says that ever since the Soviet Union fell, people have been taking Western democracy as given, but it's turning out now to not be that easy.</p>
<p>Returning to the campus topic, he notes a rise in shout-downs and dis-invitations in 2016–2017, especially after Trump's inauguration. The few examples he gave were familiar from <a href="https://www.manhattan-institute.org/diversity-delusion">The Diversity Delusion</a> that I read and published <a href="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk//articles/notes-on-the-diversity-delusion-book/">notes on</a> in 2019. Namely, the <a href="https://en.wikipedia.org/wiki/2017_Berkeley_protests">protests against Milo Yiannopoulos at Berkeley University in 2017</a> and <a href="https://www.nytimes.com/2017/03/13/opinion/understanding-the-angry-mob-that-gave-me-a-concussion.html">student mob attack on Charles Murray and Allison Stanger at Middlebury College</a>. In general, students feel it's appropriate to attack others if they don't like what they're saying.</p>
<h2 id="causes">Causes</h2>
<p>The events have left a new moral culture of "safetyism" in colleges, where students think of themselves and others as very fragile. It's often reflected in terms like:</p>
<ul>
<li>Safe spaces</li>
<li>Trigger warnings</li>
<li>Micro-aggressions</li>
<li>Bias response teams</li>
<li>"Matrices of oppression"</li>
<li>Call out culture, often for single words</li>
</ul>
<p>"Safetyism" could be linked to these phenomena:</p>
<ul>
<li>Political polarizations, with purifications of the faculty.</li>
<li>Rising anxiety and depression for Gen Z immersed in social media.</li>
<li>Paranoid parenting.</li>
<li>Decline of free play.</li>
<li>Growth of bureaucracy and moral dependence.</li>
<li>Rising passions for social justice with shift to "equal outcomes", not proportional procedural social justice.</li>
</ul>
<p>Haidt also covers three ideas that he claims, if believed, are so terrible that their follower guarantees themselves to become failures in life:</p>
<ol>
<li>What doesn't kill you makes you weaker.</li>
<li>Always trust your feelings.</li>
<li>Life is a battle between good people and evil people.</li>
</ol>
<h3 id="bad-idea-1-what-doesnt-kill-you-makes-you-weaker">Bad Idea 1: What Doesn't Kill You Makes You Weaker</h3>
<p>For this, Haidt brings up his first book "<a href="https://www.happinesshypothesis.com">The Happiness Hypothesis</a>" where he went through the literature on happiness. Every society has a resilience principle and it goes like Friedrich Nietzsche's <q>What doesn't kill me, makes me stronger</q>. In looking for a word for this principle, Haidt arrived at Nassim Nicholas Taleb and his <a href="https://en.wikipedia.org/wiki/Antifragile">Antifragile</a> book. Taleb suggests there are systems that need to be tested for strength, and came up with the term "antifragile" for things that become stronger if you put them to more use. Human bones are an example of that, which, if exercised, get stronger in proportional to the need.</p>
<p>The human immune system is another example. 1990s to 2010 saw a <a href="https://www.vox.com/2014/11/6/7163713/peanut-allergy-symptoms">tripling in peanut allergies</a>. One reason may have been the suggestion given to pregnant and nursing women to avoid peanuts under the assumption that it's exposure to peanuts that cause peanut allergies. Yet peanut allergies continued to rise. According to <a href="https://www.nejm.org/doi/full/10.1056/nejmoa1414850">"Randomized Trial of Peanut Consumption in Infants at Risk for Peanut Allergy"</a>, however, the reverse seems to be true — if you expose infants to peanuts, their learns to process them, reducing the likelihood of allergic reactions later.</p>
<p>Protectionism goes beyond peanuts, with parents hovering over and protecting their kids throughout their childhood. Haidt highlights this with a quick audience question — "at what age were you let out to roam by yourself?". That is, going alone to a friend's house or to the store to fetch candy. Audience members born before 1982, gen-x and baby boomers, responded they were out the door around the age of 6–8. Gen Z, kids born after 1994, got out by themselves far later, at ages 10–14. Just as crime was plummeting in the late 90s and 2000s and it was safer to let kids out, parents decided otherwise.</p>
<p>It's also play that has suffered, with less and less unsupervised play available to them. In play we learn how to coordinate, how to be empathetic, how to make rules, enforce them or be flexible with them. What's the effect when you take unsupervised play away from kids? Peter Gray says it leads to psychopathology in his "<a href="https://www.psychologytoday.com/files/attachments/1195/ajp-decline-play-published.pdf">Decline of Play and the Rise of Psychopathology in Children and Adolescents</a>" article.</p>
<p>Once a kid has learned how to ride the skateboard, they'll be trying to ride down staircases. Kids are seeking out risk their brain needs to evolve, and preventing them from doing so may actually harm them:</p>
<blockquote>
<p>Trying to eliminate all such risks from children’s lives may be dangerous. There may be a psychological analog to the hygiene hypothesis. In the same way, by shielding children from every possible risk, we may lead them to react with exaggerated fear to situations that aren't risky at all and isolate them from the adult skills that they will one day have to master.
— <cite>Alison Gopnik</cite></p>
</blockquote>
<p>Good concrete examples can be shown in the evolution of outdoor parks. Whereas they may have started out being occasionally dangerous enough for kids' lives to be at risk, they've morphed past the optimal point of safety into spaces that provide too few possibilities of getting hurt. And it's the possibility of getting hurt that creates confident and able people out of kids.</p>
<p>So in the light of today's restrictive plastic parks...</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/playground-too-safe.jpg"><img src="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/playground-too-safe.jpg" alt="" /></a></p>
</figure>
<p>...we perhaps should be moving back to more exciting ones:</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/playground-just-right.jpg"><img src="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/playground-just-right.jpg" alt="" /></a></p>
</figure>
<p>As the dictum goes:</p>
<blockquote>
<p>Prepare the child for the road, not the road for the child.</p>
</blockquote>
<p>Britain's said to be ahead of the US in that respect, as described in "<a href="https://www.nytimes.com/2018/03/10/world/europe/britain-playgrounds-risk.html">In Britain’s Playgrounds, ‘Bringing in Risk’ to Build Resilience</a>".</p>
<blockquote>
<p>It's about exploring controlled risk, risk that we’ve carefully designed. We've got the gorse bushes, which are quite spiky. The child will touch it and learn it is a spiky bush.
— <cite>Chris Moran, manager of Queen Elizabeth Olympic Park</cite></p>
</blockquote>
<p>In America, however, it's believed that if anyone gets hurt, it's to be banned, leading to ridiculous bans like "<a href="https://www.nationalreview.com/2018/08/university-bans-snowball-fights-and-water-guns/">University Bans Snowball Fights and Water Guns</a>".</p>
<p>Haidt recently heard of a meme that would be awful to teach to kids:</p>
<blockquote>
<p>We are all balloons filled with feelings in a world full of pins.</p>
</blockquote>
<p>He then contrasts it with Taleb's metaphor of the candle and a fire, whereas a candle must be protected lest the flame go out, a fire grows stronger with more wind. <q>You want to be the fire and wish for the wind</q>.</p>
<h3 id="bad-idea-2-always-trust-your-feelings">Bad Idea 2: Always Trust Your Feelings</h3>
<p>The second dangerous idea is overturned by most cultures and self-help literature, be it Stoics, Buddhists or Hindus.</p>
<blockquote>
<p>What really frightens and dismays us is not external events themselves, but the way in which we think about them. It is not things that disturb us, but our interpretation of their significance.
— <cite>Epictetus</cite></p>
</blockquote>
<blockquote>
<p>Your worst enemy cannot harm you as much as your own thoughts, unguarded. But once mastered, no one can help you as much, Not even your father or your mother.
— <cite>Buddha</cite></p>
</blockquote>
<p>This is also the mechanism behind cognitive behavioral therapy. If you can get people to question their thinking patterns, they could be freed from their depression. Cognitive behavior therapy teaches you to become aware of common cognitive distortions or biases, among which are:</p>
<ul>
<li>Negative filtering — only notice the bad stuff.</li>
<li>Over-generalizing</li>
<li>Dichotomous thinking — binary, black and white thinking.</li>
<li>Emotional reasoning</li>
</ul>
<p>These biases are encouraged in students on American campuses, by, for example, when they're taught about micro-aggressions. While originally perhaps a useful term to refer to intentional acts of minor hostility, it has now become a term to refer to mere social missteps (<i>faux pas</i>) or subjectively applied to innocent yet often heard questions, like "where are you from?". When so many daily interactions are now labeled as micro-aggressions, you get kids who are hyper-sensitive to interpret unintentional general comments egocentrically as comments about them — balloons in a world of pins. They're beginning to ignore basic aspects of morality, like judging actions by intent.</p>
<p>Haidt says that more faux pas are inevitable when diversity increases. There are two ways to handle that:</p>
<ol>
<li>Teach students to be more polite and give everyone the benefit of the doubt.</li>
<li>Teach students to focus on emotional impact, ignore intent, interpret ambiguous acts as acts of aggression and report each other.</li>
</ol>
<p>The latter is what Haidt fears American campuses are doing.</p>
<h3 id="bad-idea-3-life-is-a-battle-between-good-people-and-evil-people">Bad Idea 3: Life is a Battle Between Good People and Evil People</h3>
<p>We are all prone to tribalism and dichotomous thinking.</p>
<blockquote>
<p>If only it were all so simple! If only there were evil people somewhere insidiously committing evil deeds, and it were necessary only to separate them from the rest of us and destroy them. But the line dividing good and evil cuts through the heart of every human being.
— <cite>Александр Солженицын (Aleksandr Solzhenitsyn)</cite></p>
</blockquote>
<p>When you put human tribalism tendencies together with an idea of dichotomies, you get a call-out culture, where before noticing mistakes in yourself, you point them out in others. Kira Barrett described that in her "<a href="https://web.archive.org/web/20190315123106/http://www.thesmithsophian.com/walking-on-eggshells-how-political-correctness-is-changing-the-campus-dynamic/">Walking on Eggshells — How Political Correctness is Changing the Campus Dynamic</a>" article:</p>
<blockquote>
<p>During my first days at Smith, I witnessed countless conversations that consisted of one person telling the other that their opinion was wrong. The word “offensive” was almost always included in the reasoning. Within a few short weeks, members of my freshman class had quickly assimilated to this new way of non-thinking. They could soon detect a politically incorrect view and call the person out on their “mistake.” I began to voice my opinion less often to avoid being berated and judged by a community that claims to represent the free expression of ideas. I learned, along with every other student, to walk on eggshells for fear that I may say something “offensive.” That is the social norm here.
— <cite>Kira Barrett</cite></p>
</blockquote>
<p>A big reason Haidt believes is intersectionality. He agrees that the core idea is right — identities don't just add, they interact — being a black woman is not just the sum being black and being a woman. His problem with intersectionality is how it's taught on campuses and interpreted.</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/intersectionality.jpg"><img src="https://m811.com/articles/notes-on-the-three-terrible-ideas-weakening-gen-z-and-damaging-universities-and-democracies-talk/intersectionality.jpg" alt="" /></a></p>
</figure>
<p>Haidt claims intersectionality is taught with identities being binary — you're either on one side or another — and the identities themselves are morally valenced. The ones above the horizontal line are privileged and morally oppressors. Ones below the line are the unprivileged, oppressed and therefore good people. The quintessential oppressors are be those at the top — heterosexual, white and male.</p>
<h2 id="summary">Summary</h2>
<p>Haidt ends with corrections to the three dangerous ideas:</p>
<ol>
<li><s>What doesn't kill you makes you weaker.</s>
Prepare your child for the road, not the road of the child.</li>
<li><s>Always trust your feelings.</s>
It is not things that disturb us, but our interpretation of their significance.</li>
<li><s>Life is a battle between good people and evil people.</s>
The line diving good and evil cuts through the heart of every human being.</li>
</ol>
<p>We should <q>welcome viewport diversity and the clash of ideas</q>.</p>
<blockquote>
<p>I believe that learning at its best is the antithesis of comfort. […]
The collision of views and ideologies is in the DNA of the academic enterprise.
— <cite>Ruth Simmons</cite></p>
</blockquote>
<p>Haidt also suggests learning more about cognitive behavioral therapy from <a href="http://sethgillihan.com/books/cognitive-behavioral-therapy-made-simple/">Cognitive Behavioral Therapy Made Simple</a>.</p>
<h2 id="q--a">Q & A</h2>
<h3 id="bullying">Bullying</h3>
<p>In the questions and answers section Haidt clarified the concept-creep of the word "bullying". In original literature bullying was said to be defined by three qualities:</p>
<ul>
<li>Power differential</li>
<li>Threat of actual violence</li>
<li>Prolonged</li>
</ul>
<p>These days however every confrontation is seen as bullying. Haidt's suggestion is to give kids more unsupervised time and to not prevent them from solving disputes by themselves. It's only multi-day tormenting you want to prevent.</p>https://m811.com/articles/notes-on-simple-rules-book/Notes on «Simple Rules» book2019-09-18T13:05:00.000Z2019-09-18T13:05:00.000Z<big>
The books claims that in a world filled with an abundance of information and not enough time to deliberate, coming up with simple rules helps future decision making by providing a good balance of consistency and flexibility. In other words, <q>simple rules allow people to act without having to stop and rethink every decision</q>. In group settings, shared rules help orient and synchronize people in new situations. They also let people efficiently rely on the wisdom of and avoid mistakes of their peers.
</big>
<h2 id="complex-world">Complex World</h2>
<p>Authors start with a few stories and examples of the complexity of the modern world.</p>
<blockquote>
<p>Complexity arises whenever a system—technical, social, or natural—has multiple interdependent parts.</p>
</blockquote>
<blockquote>
<p>Consider how central bankers responded to increased complexity in the global banking system. In 1988 bankers from around the world met in Basel, Switzerland, to agree on international banking regulations, and published a 30-page agreement (known as Basel I). Sixteen years later, the Basel II accord was an order of magnitude larger, at 347 pages, and Basel III was twice as long as its predecessor. When it comes to the sheer volume of regulations generated, the U.S. Congress makes the central bankers look like amateurs. The Glass-Steagall Act, a law passed during the Great Depression, which guided U.S. banking regulation for seven decades, totaled 37 pages. Its successor, Dodd-Frank, is expected to weigh in at over 30,000 pages when all supporting legislation is complete.</p>
</blockquote>
<p>The exponential growth in legislation reminds me of libertarian critique of government regulation — for every new regulation, special interest groups arise demanding exceptions. This quickly results in taxes, tax exemptions, tax exemption exceptions ad infinitum, a spiraling resource waste on bureaucracy at the cost of actual value.</p>
<blockquote>
<p>The policies governing U.S. income taxes totaled 3.8 million words as of 2010. […] Such an exhaustive tome should leave nothing to chance. And yet, when forty-five tax professionals were given identical data to calculate one fictional family’s tax bill, they came up with forty-five different estimates of the couple’s tax liability, ranging from <span class="math math-inline"><span class="katex"><span class="katex-mathml"><math xmlns="http://www.w3.org/1998/Math/MathML"><semantics><mrow><mn>36</mn><mo separator="true">,</mo><mn>322</mn><mi>t</mi><mi>o</mi></mrow><annotation encoding="application/x-tex">36,322 to </annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.8388800000000001em;vertical-align:-0.19444em;"></span><span class="mord">36</span><span class="mpunct">,</span><span class="mspace" style="margin-right:0.16666666666666666em;"></span><span class="mord">322</span><span class="mord mathnormal">t</span><span class="mord mathnormal">o</span></span></span></span></span>94,438. The tax code is so confusing, even IRS experts give the wrong advice one time out of three.</p>
</blockquote>
<p>Regulation that no single person can understand eventually ends becoming a weapon. It's inevitable that people make unlawful steps sooner or later, and with a noose hanging around everyone's neck, it's a matter of just picking who to prosecute.</p>
<blockquote>
<p>People often attempt to address complex problems with complex solutions. For example, governments tend to manage complexity by trying to anticipate every possible scenario that might arise, and then promulgate regulations to cover every case.</p>
</blockquote>
<p>Complicated solutions to all complex problems are exactly what the authors try to avoid, claiming that simple heuristics outperform more complex ones in practice. Take company strategy, for example:</p>
<blockquote>
<p>Don and his colleague Rebecca Homkes conducted a survey of managers in over 250 companies around the world. When asked to list their company’s top priorities for the next few years, only half of the managers could name even their company’s single most important objective.</p>
</blockquote>
<p>Corporate strategy is better relayed to employees through simple rules they can apply easily than through obtuse manuals too time consuming to consult. Using simple rules need not necessarily result in simplistic behavior. A colony of ants, following very basic coordination rules, end up with quite advanced behavior of obstacle avoidance and resource gathering.</p>
<h2 id="essence-of-simple-rules">Essence of Simple Rules</h2>
<blockquote>
<p>Michael Pollan distilled his nutritional insights into three simple rules: “Eat food. Not too much. Mostly plants.” By “food” Pollan means real food—vegetables, fruits, nuts, whole grains, and meat and fish—rather than what he calls “edible food-like substances” found in the processed-food aisles of the grocery store.</p>
</blockquote>
<p>Pollan's principles follow the four traits of simple rules:</p>
<ol>
<li><q>First, they are limited to a handful.<br />
Capping the number of rules makes them easy to remember and maintains a focus on what matters most.</q></li>
<li><q>Second, simple rules are tailored to the person or organization using them.</q></li>
<li><q>Third, simple rules apply to a well-defined activity or decision, such as prioritizing injured soldiers for medical care.</q></li>
<li><q>Finally, simple rules give concrete guidance without being overly prescriptive.</q></li>
</ol>
<p>Limiting the number of rules plays to the <a href="https://en.wikipedia.org/wiki/Pareto_principle">Pareto principle</a> (or power law distribution), which states that the majority of effects emerge from a minority of causes. An extraneous long tail would have diminishing returns. A handful of rules, preferably from 3 to 5, also fit well in the 4±2 items our memories are said to handle at once. The more rules there are, the more ways they have to interact or contradict each other. Being able to think about them together may highlight inconsistencies that with longer lists could remain hidden.</p>
<p>In contrast to Pollan's eating rules above, rules for a college football team should be different from rules for the average dieter, highlighting how simple rules should be specific to a particular group or type of person.</p>
<blockquote>
<p>[…] Turley’s rules for his players are “Eat breakfast,” “Stay hydrated,” and “Eat as much as you want of anything that you can pick, pluck, or kill.” These rules make a lot of sense for very large and very active college-football players, who have a tendency to stay up late and get up late, exercise voraciously, and easily work off the calories.</p>
</blockquote>
<p>Aside from being tailored to specific people, rules should also be unique to particular activities. If they cover too broad situations and contexts, they become platitudes or clichés.</p>
<h3 id="consistency-vs-flexibility">Consistency vs Flexibility</h3>
<blockquote>
<p>A hammer is just the thing for nails, but useless when it comes to sawing a plank. The same is true of simple rules: to be effective, they must fit the task at hand. Simple rules work best when flexibility matters more than consistency.</p>
</blockquote>
<p>Simple rules aren't always the best way to approach decision making. Environments that require a rote activity or where lives are on the line, checklists form a better guide. While airline maintenance and medical procedures benefit more from precise checklists, <a href="https://en.wikipedia.org/wiki/Triage">medical triage</a>, where time is priority, is better served by heuristics.</p>
<p>There's a risk of <a href="https://en.wikipedia.org/wiki/Overfitting">overfitting</a> with any model of the world and rules derived from it. An overfitted model is one that followed historical data and its errors too precisely, missing the underlying trend and assuming the future will look <em>exactly</em> like the past. It can't handle additional historical data without fail, let alone predict the future in any accuracy. Overfitting shows up in laws and regulations that were relevant for a short period and later become useless or worse, dangerous.</p>
<blockquote>
<p>In very complex systems, like the stock market or the economy as a whole, where causal relations are poorly understood and shift over time, the risks of overfitting past data are particularly acute. Statisticians have found that complicated models consistently fail to outperform simple ones in forecasting economic trends, and the accuracy of their predictions has not improved over time. When it comes to modeling complex systems, sophisticated does not always equal effective.</p>
</blockquote>
<p>Simple rules avoid overfitting by not being <em>too</em> specific.</p>
<blockquote>
<p>Counter-intuitive as it may sound, simple rules can outperform more analytically complicated and information-intensive approaches even when there is ample time and information to make a decision. This is especially true in situations when the links between cause and effect are poorly understood, when important variables are highly correlated, when a few factors matter most, and when a gap exists between knowing what to do and actually doing it. Simple rules do not trump complicated models every time, but they do so more often than you might think.</p>
</blockquote>
<p>Given the possibility, it seems people voluntarily end up considering peripheral details at the expense of more important ones:</p>
<blockquote>
<p>One study, for example, asked college students to consider several factors (such as the required reading, syllabus, prerequisites, and past students’ ratings) and weigh their relative importance before deciding which courses to take. A control group was given the same information and simply asked to choose their courses. The students who gave less thought to the variables tended to focus on a single variable—past students’ ratings—in making their decisions. In contrast, their classmates who considered more variables actually made worse decisions (as measured by their ultimate satisfaction with the courses they took). By ruminating on unimportant details, the students failed to give the most important variables the weight they deserved. Simple rules minimize the risk of overweighing peripheral considerations by focusing on the criteria most crucial for making good decisions.</p>
</blockquote>
<p>Simple rules, being relatively easy to remember and follow, are also more likely be used than more comprehensive guidelines, as highlighted by another study:</p>
<blockquote>
<p>They participated in a study to learn basic accounting to run their businesses better. Each entrepreneur was randomly assigned to one of three experimental groups. The first group studied accounting the way it is taught in most universities, as a complicated body of knowledge to be mastered. A second group studied accounting as simple rules like “Keep personal and business money in different drawers” and “Only transfer money from one drawer to the other with a written IOU.” The third group received no accounting instruction. The entrepreneurs who learned the simple rules of accounting were more likely to translate their knowledge into action. They improved their bookkeeping and cash management and also increased sales by 25 percent. In contrast, the entrepreneurs who were exposed to accounting as a complicated body of knowledge were no better off than those who were not taught any accounting at all.</p>
</blockquote>
<p>Simplicity plays an even bigger role when people are tired or stressed:</p>
<blockquote>
<p>One study of dieting, for example, compared perseverance and weight loss on a simple diet plan versus a complicated one. While the dieters who stuck with the program lost weight on both plans, people were more likely to adhere to the simple diet and abandon the complicated one. When asked why they quit, the lapsed dieters cited complexity as the single most important reason for giving up.</p>
</blockquote>
<h2 id="decision-rules">Decision Rules</h2>
<p>Authors describe three rule types that are often used in decision making.</p>
<ol>
<li><strong><a href="https://m811.com/articles/notes-on-simple-rules-book/#boundary-rules">Boundary rules</a></strong> help choose among mutually exclusive options.</li>
<li><strong><a href="https://m811.com/articles/notes-on-simple-rules-book/#prioritizing-rules">Prioritizing rules</a></strong> help choose where to direct resources.</li>
<li><strong><a href="https://m811.com/articles/notes-on-simple-rules-book/#stopping-rules">Stopping rules</a></strong> help choose when to reverse a decision or cancel an action.</li>
</ol>
<h3 id="boundary-rules">Boundary Rules</h3>
<p>Boundary rules, like when to go forward with a deal, help narrow down and choose between mutually exclusive options. Limited not only to professionals, boundary rules are used even by burglars:</p>
<blockquote>
<p>But most real-life burglars rely on only a few simple rules to decide if a house is unoccupied, and hence a good candidate for breaking and entering.</p>
<p>The burglars significantly outperformed chance in identifying occupied houses, and two-thirds of the time they used a single rule to pick their target: “Avoid houses with a vehicle parked outside.” This simple rule is unlikely to dazzle fans of heist films like Ocean’s Eleven, but it works. The presence of a vehicle was indeed the single most reliable predictor of occupancy in the homes that the researchers photographed.</p>
</blockquote>
<p>When it comes to quick decision making, nothing beats having a heuristic in place to quickly disqualify options. They turn out to be handy even in psychological evaluations, identifying depressed patients 97% of the time if they answer yes to all four questions:</p>
<blockquote>
<p>A recently developed diagnostic tool for depression relies on four simple rules, expressed as questions that can be asked in under a minute: Have you cried more than usual within the last week? Have you been disappointed in yourself or hated yourself within the last week? Have you felt discouraged about the future within the last week? Have you felt that you failed in your life within the last week</p>
</blockquote>
<h3 id="prioritizing-rules">Prioritizing Rules</h3>
<p>With boundary rules help pick options to qualify, prioritizing rules help distribute finite resources among alternatives. They help resolve bottlenecks, where the amount of opportunities exceed resources, by helping focus on what can have the greatest impact.</p>
<h3 id="stopping-rules">Stopping Rules</h3>
<p>In decision theory circles, the best known stopping problem is the <a href="https://en.wikipedia.org/wiki/Secretary_problem">secretary problem</a>. Suppose you need to hire the best secretary out of all the candidates. You also get a single interview with every candidate and have to decide then and there whether to hire or move to the next. Who do you select, or in other words, when you stop searching and settle down? If you pick a candidate too quickly, you're likely to miss out on later better candidates. If you hesitate for too long, you might pass on the best candidate of the group and have no choice but to settle on a worse candidate.</p>
<blockquote>
<p>Whenever alternatives present themselves sequentially (as opposed to appearing all at once), the question of when to stop searching and make a choice arises.</p>
</blockquote>
<p>Another manifestation of the stopping problem is mate selection:</p>
<blockquote>
<p>Biologists have observed several different rules that insects use to decide when to end their search and settle on a mate. These rules include “Choose a mate who meets your quality threshold,” which is known as the <strong>fixed-threshold strategy</strong>. In contrast, the female crickets that lower the bar if they do not run across enough high-quality males adhere to what is known as the <strong>variable-threshold strategy</strong>. […] Another simple rule for ending a mate search is to visit a fixed number of potential mates and then return to the highest-quality mate within that sample. And anyone who has stayed until last call at a singles bar will recognize the “fixed threshold with last chance option” rule, where the seeker maintains his or her threshold until a set time, and then mates with the next potential partner regardless of quality.</p>
</blockquote>
<p>Sometimes it's not about stopping search, but instead stopping an ongoing engagement, be it financial or professional.</p>
<blockquote>
<p>A well-documented tendency among human decision makers, known as the status quo bias, leads individuals to hold ’em when they should fold ’em across a range of decisions.</p>
</blockquote>
<blockquote>
<p>Stopping rules can help investors by providing some guidance on when to sell their assets. And, if there was ever a good time to know when to sell a stock, it was right before the Great Depression.</p>
<p>Out of the rubble of the 1929 crash, one banker rose up stronger than ever. Gerald Loeb, the son of a French wine merchant, sniffed out the crash of 1929 before it happened and helped his clients avoid heavy stock market losses.</p>
<p>The secret weapon of Loeb’s investing strategy was a simple but powerful stopping rule: “If an investment loses 10 percent of its initial value, sell it.” This rule ensures that the investor does not stick with a loser over the long term. While it may be tempting to wait for a pet stock to regain its value, Loeb’s 10 percent rule recognizes that it is often best to cut your losses and move your money elsewhere.</p>
</blockquote>
<p>Being able to walk away is especially vital in situations where people have a tendency to double down:</p>
<blockquote>
<p>Escalating commitment to a failed course of action is a well-documented error, with over 150 studies of cases as diverse as NBA teams overplaying draft-pick busts, rogue traders doubling down on their money-losing investments, construction projects becoming money pits, and failing military campaigns where success is always described as just around the corner (the Vietnam War is often used as the poster child of escalating commitment to a failed course of action).</p>
</blockquote>
<h2 id="process-rules">Process Rules</h2>
<p>With decision rules covering what to do, process rules cover how to do something. Here, too, three are brought out:</p>
<ol>
<li><strong><a href="https://m811.com/articles/notes-on-simple-rules-book/#how-to-rules">How-to rules</a></strong> cover basic task execution.</li>
<li><strong><a href="https://m811.com/articles/notes-on-simple-rules-book/#coordination-rules">Coordination rules</a></strong> cover and handle multiple people working towards one goal.</li>
<li><strong><a href="https://m811.com/articles/notes-on-simple-rules-book/#timing-rules">Timing rules</a></strong> cover schedules and deadlines.</li>
</ol>
<h3 id="how-to-rules">How-To Rules</h3>
<blockquote>
<p>How-to rules address the basics of getting things done without prescribing every detail of what to do.</p>
</blockquote>
<p>Being adaptable makes simple how-to rules fit novel situations far better than precise checklists. How-to rules also provide direction when approaching creative tasks.</p>
<blockquote>
<p>By constraining infinite possibilities, simple rules allow creativity to flourish, less from thinking outside the box and more from deciding how to draw the box in the first place.</p>
</blockquote>
<p>This was the musical strategy of White Stripes:</p>
<blockquote>
<p>“The whole point of the White Stripes,” according to founder and frontman Jack White, “is the liberation of limiting yourself.” Their breakout album, 2001’s White Blood Cells, which is featured on many lists of the decade’s best albums, follows five simple rules: (1) no blues; (2) no guitar solos; (3) no slide guitar; (4) no covers; and (5) no bass. These rules constrained the band to a box—but it was their box, and staying in that box helped enable their rapid-fire creativity. “I’m disgusted by artists or songwriters who pretend there are no rules,” Jack White said in a New York Times interview. “There’s nothing guiding them in their creativity. We could’ve spent six months making our last album. We could have recorded 600 tracks. Instead, we went and made the whole album, 18 songs, in 10 days.”</p>
</blockquote>
<p>As with decision rules, authors suggest there's a balance between having no rules and having too many:</p>
<blockquote>
<p>The firms that eschewed rules altogether were soon overcome by complexity, dissipating their scarce time, attention, and cash in the pursuit of any opportunity that passed their way. But the firms that followed many rules were too slow and stodgy. The environment of these technology-based companies was far too complex for a structured solution, enumerated in a fat rule book, to ever make sense.</p>
</blockquote>
<h3 id="coordination-rules">Coordination Rules</h3>
<p>Flight of birds is an example of coordination rules, where a simple algorithm guides each bird results in complex group behavior. Coordination rules describe behavior in relation to others, even if those others are distant in time or space.</p>
<p>Groups can achieve objectives that are out of reach for a single individual. Predators, for example, can hunt prey many times their size if they manage to coordinate their attack. Even insects.</p>
<blockquote>
<p>A solitary locust is nothing special. But as a swarm, locusts constitute a pestilence of, well, biblical proportions. Even today, locusts threaten the sustenance of one in every ten people on earth. A swarm can contain millions of locusts, blacken the sky over several kilometers, and travel the length of a continent, devastating crops and livestock along the way. Yet most of the time locusts live a solitary existence, avoiding contact with one another, and only occasionally mobilize into a hellish army.</p>
</blockquote>
<p>Improvisation is an activity that benefits from coordination.</p>
<blockquote>
<p>The best-known rule is to build on whatever is said or done just beforehand by saying, “Yes, and . . .” Another rule is “Don’t tell jokes,” because they often stifle an emerging storyline by imposing an artificial punch line onto an organic situation. The rule to make others look good underscores the importance of helping other players shine. In improv, prima donnas just make everyone, including themselves, look second-rate. Instead, you help yourself by helping the group.</p>
</blockquote>
<p>Some situations require rules to coordinate across time and distance:</p>
<blockquote>
<p>To foster battlefield coordination, Napoleon is reputed to have issued a standing order to “march toward the sound of gunfire,” a simple rule that enabled his officers to coordinate their activities without knowing exactly what was happening. Generals and soldiers could locally adapt to the facts on the ground, such as deteriorating weather, a gap in enemy defenses, or unexpectedly intense resistance, which were impossible to anticipate. But the rule also helped ensure that the fighting force would arrive where it was most needed and would have the most impact.</p>
</blockquote>
<h3 id="timing-rules">Timing Rules</h3>
<p>Where how-to and coordination rules dictate what and who should do something, timing rules help decide when to act. One problem that benefits from timing rules is insomnia, whose sufferers may be able to alleviate their symptoms by following these four simple rules for when to sleep:</p>
<ol>
<li><q>“Get up at the same time every morning,” which turns out to be more crucial than a regular bedtime for establishing a restful sleep pattern.</q></li>
<li><q>“Avoid going to bed until you feel sleepy,” even if this means hitting the hay later than you would ideally like.</q></li>
<li><q>“Do not stay in bed if you are not sleeping,”</q></li>
<li><q>“Reduce the time spent in bed.”</q></li>
</ol>
<blockquote>
<p>Following these rules keeps troubled sleepers from spending ten or twelve hours in bed in order to snag a few hours of sleep—a common pattern in older adults. In the short run, these rules may lead to people feeling tired or sleep deprived as they adjust. But over the long term, patients who follow these rules usually enjoy a deeper, more restful sleep that comes on more quickly.</p>
</blockquote>
<p>Timing rules can be split into two:</p>
<ul>
<li><strong>Event pacing</strong>, where an external event triggers an action, like going to bed only when tired.</li>
<li><strong>Time pacing</strong>, where the trigger lies in the time of day or deadline, such as a daily alarm clock.</li>
</ul>
<p>Out of all rule types you might use, timing rules may often surface last — <q>knowledge about time requires experiencing enough events over a sufficiently long period to recognize sequences of action or particular rhythms that make sense to use.</q></p>
<p>Dragonflies follow a few timing rules to guide their migration:</p>
<blockquote>
<p>The scientists also discovered that a few timing rules explained when the insects fly and when they stay put. One rule is to fly only when the nighttime temperature falls for two consecutive nights. Falling nocturnal temperatures are highly correlated with frigid northern winds that carry the green darners south and remind the dragonflies to get going. While some wind is ideal, extreme gusts are dangerous regardless of which way the wind is blowing. So the second timing rule is to stay put on windy days—that is, when the wind blows more than fifteen miles per hour. Together, these timing rules specify the events that trigger when green darners start flying.</p>
</blockquote>
<h2 id="source-of-rules">Source of Rules</h2>
<p>The book enumerates a few sources of inspiration for coming up with simple rules:</p>
<ol>
<li><strong>Evolution</strong>, where natural selection or culture picks the best and worse rules for us.</li>
<li><strong>Personal experience</strong>, where you rely on your own career.</li>
<li><strong>Others' experience</strong>, where you rely on others' mistakes.</li>
<li><strong>Scientific experience</strong>, where you rely on the scientific method more than personal anecdotes.</li>
</ol>
<h3 id="evolution">Evolution</h3>
<p>It's not always people that need to come up with simple rules to get tasks done. Butterflies have a particularly important problem they need to solve, and that's where to meet:</p>
<blockquote>
<p>Butterflies have a lot of space to cover, and very little time to do it. If butterflies were the size of humans, their population density would be one-tenth that of Alaska. Butterflies live, on average, just one month, so time is of the essence. The butterflies need to find each other, and quickly. From an evolutionary perspective, the stakes are high, since romantics who fail to mate cannot pass their genes on to the next generation.</p>
</blockquote>
<p>After 50 millions years of speed dating and missed appointments, they converged on a few simples rules of their own:</p>
<blockquote>
<p>First, “Fly uphill most of the time.” Second, “Fly toward the highest slope in sight” (within the butterfly’s range of vision of about 165 feet). Third, “Pause to check out local peaks, even if they are not the highest, but leave if you do not get lucky right away.” Fourth, “Periodically make a random movement” to avoid getting trapped on a summit that is the highest peak in the nearby vicinity, but not the highest overall.</p>
</blockquote>
<p>Some rules evolve not through natural selection, but in human communities through changes in culture.</p>
<blockquote>
<p>Joke stealing is the cardinal sin in standup comedy today, but it was not always so. For much of the twentieth century, stealing jokes was no big deal. In the vaudeville era, performers would repeat other comedians’ material verbatim without attributing the source. Later comedians, such as Milton Berle and Bob Hope, drew on vast stores of generic jokes. Expert delivery and timing, rather than originality, mattered most.</p>
<p>For example, comedians often formulate material collaboratively. In this instance, the rule gives ownership of the joke to the person who came up with the premise, rather than the punch line. This rule arose through discussions among comedians and spread. When two performers come up with similar jokes, the rule is that the first person to perform the joke on television owns it, giving authorship to the comedian who can prove he or she told the joke first. Another ownership issue, what to do when one comic sells a joke to another, or writes the joke as the comic’s employee, is resolved by the rule that the person who paid for a joke owns it, and the originator cannot publicly identify herself as the joke’s writer.</p>
</blockquote>
<p>While rules that have evolved through evolution often end up elegant and simple, being ingrained, they don't lend themselves to easy adaption if circumstances change. That's when people should turn to more concious rule making.</p>
<h3 id="personal-experience">Personal Experience</h3>
<p>As rules are intended to be context and person specific, getting inspiration from personal experience is a good way to create relevant simple rules. Personal rules are also more likely to be adhered to.</p>
<p>The second best source is personal experience of people around you:</p>
<blockquote>
<p>The experience of others can be as valuable as one’s own experience in shaping people’s simple rules. Advice from experienced people about how to parent, play poker, or manage people, for example, whether given directly or conveyed through a book or magazine article, can prove useful in formulating one’s own approach. Personal observation is another effective way to learn what works for others. We also use analogies, finding similarities between our own experience and others’, which can streamline the process of devising our rules.</p>
</blockquote>
<p>They have a few suggestions on eliciting rules:</p>
<blockquote>
<p>When talking with your role models, it’s important to recognize that their simple rules will most likely be implicit, so asking for a list of rules may not be the best approach. A few tactics can help surface tacit rules. First, you can explain how you manage your identified bottleneck, and ask them what they do differently. It’s also productive to tease out extremes, by asking if there are things that they always do or never do when managing the target activity. Another way to explore your role model’s tacit rules is to ask them to walk you through a few recent decisions—what they did and why. People typically find it easier to describe their rules in the context of concrete examples rather than in abstract terms.</p>
</blockquote>
<p>"Others" don't even have to be humans. The book describes the designing of the Tokyo commuter-rail system for which its designers drew inspiration from biology. To figure out ways to best link up Tokyo with its 36 nearby towns, they turned to many-headed slime mold. By placing food to stand in for cities and lights, which the mold avoids, to stand in for mountains, lakes and other inaccessible places, they could investigate the methods evolution has come up with for network design.</p>
<blockquote>
<p>From their observations, the team codified the how-to and stopping rules for the slime’s expansion, which included: (1) begin by searching randomly in many directions for food; (2) when you find food, thicken the tube; and (3) when you don’t find food, shrink the tube.</p>
</blockquote>
<p>Mold isn't the only useful biological source of inspiration. Social insects, like termites and bees, have long used simple rules from which interesting behavior arises.</p>
<blockquote>
<p>They have enough collective intelligence to accomplish complex tasks like finding nests or migrating long distances, but since each animal has little brainpower and few physical skills, their actions can often be captured by simple rules.</p>
</blockquote>
<h3 id="scientific-experience">Scientific Experience</h3>
<p>More methodological approaches to experience, with quantifiable data, gets us to deriving rules from scientific evidence.</p>
<blockquote>
<p>Scientific evidence, however, is not inherently simple. The resulting knowledge is often full of qualifications and contingencies that hold under some circumstances, but not others. One way to develop simple rules is to review a body of scientific research, sort through to determine the most consistent findings across studies, and distill these findings into a few simple rules.</p>
</blockquote>
<p>A distillation of scientific results to simple rules also benefits less scientifically minded people:</p>
<blockquote>
<p>It prevents scientists and laypeople from speaking past one another. Marine ecologists, like all scientists, prefer to acknowledge the limits of their findings and qualify their recommendations. But to the layperson unfamiliar with scientific discourse, these uncertain statements sound like so much waffling. By focusing on the most consistent findings, scientists can be more comfortable advocating them, without qualification, as practical advice. Nonscientists receive what they want—clear guidance on action.</p>
</blockquote>
<h2 id="creating-rules">Creating Rules</h2>
<p>A general strategy the authors suggested for coming up with simple rules followed three steps:</p>
<ol>
<li><q>Figure out what will move the needles.</q></li>
<li><q>Choose a bottleneck.</q></li>
<li><q>Craft the rules.</q></li>
</ol>
<p>In a group setting, they warned against appeal to authority.</p>
<blockquote>
<p>Developing rules from the top down is a big mistake. When leaders rely on their gut instincts, they overemphasize recent events, build in their personal biases, and ignore data that doesn’t fit with their preconceived notions. It is much better to involve a team, typically ranging in size from four to eight members, and use a structured process to harness members’ diverse insights and points of view. When drafting the dream team to develop simple rules, it is critical to include some of the people who will be using them on a day-to-day basis.</p>
</blockquote>
<p>What do you do when people around you can't come to an agreement? You turn to negotiation.</p>
<blockquote>
<p>[…] negotiating simple rules works particularly well when it is cumbersome or impossible to resolve conflicts by escalating them to a higher authority. James Buchanan, a Nobel Prize–winning economist, argued that when there are competing interests, it is critical for stakeholders to negotiate decision rules before haggling over specific decisions.</p>
</blockquote>
<p>People that participated in the rule making process are far more likely to accept the consequences of using the rules, even if that means they experience loss.</p>
<blockquote>
<p>Having users make the rules confers several advantages. First, they are closest to the facts on the ground and best positioned to codify experience into usable rules. Because they will make decisions based on the rules, they can strike the right balance between guidance and discretion, avoiding rules that are overly vague or restrictive. Users can also phrase the rules in language that resonates for them, rather than relying on business jargon. By actively participating in the process, users are more likely to buy into the final rules and therefore apply them in practice. Firsthand knowledge also makes it easier to explain the rules, and their underlying rationale, to colleagues who did not participate in the process.</p>
</blockquote>
<p>After coming up with your own set of rules, it's important to test them both against historical data to see if they actually match the examples they were derived from and against future opportunities.</p>
<h3 id="improving-rules">Improving Rules</h3>
<blockquote>
<p>Although the systematic process laid out in the last two chapters can be a big help, initial rules are often automatic, obvious, and generally weak. Over time, three things happen. First, their content shifts from superficial and convenient rules to strategic and abstract ones that prove more effective over a broader range of activities and decisions. Second, the different types of rules are learned in a specific sequential order. Boundary and how-to rules usually come first, while the other rule types follow and are harder to learn. Third, the rules go through simplification cycling in which their number grows, and then shrinks and becomes constant. Over time, the rules may continue to shift as circumstances change, but the best rule users keep the number small.</p>
</blockquote>
<p>Authors also briefly touch on a general learning strategies:</p>
<blockquote>
<p>The first, specialized practice, mirrors the old saying “Practice makes perfect,” and repeats the same activity. The second, unrelated experience exploits what is known as massed practice, in which individuals take periodic breaks to do something completely different from what they are trying to learn. This strategy provides time to consolidate knowledge before moving on. The third strategy is related experience, which is what Shannon did by coaching different sports and Raghu did by practicing different kinds of poker playing.</p>
</blockquote>
<p>Based on an experiment that put the three strategies above against each other, it turned out the best strategy was the third.</p>
<blockquote>
<p>So while very specialized practice might seem better for improving because it focuses time and energy on one task, it lacks the informative contrasts that enrich understanding. While unrelated activities may offer a helpful break, they are not germane to what is being learned. Instead, people improve most rapidly with varied but related experiences.</p>
</blockquote>
<h3 id="obstacles">Obstacles</h3>
<p>With all the benefits of simple rules, why aren't they more used?</p>
<blockquote>
<p>The first obstacle is the effort required to develop simple rules. Like most worthwhile endeavors, it takes time and energy to get them right. The process of developing simple rules requires ruthless prioritization—honing in on the essential and decluttering the peripheral.</p>
</blockquote>
<p>As briefly touched on at the beginning, people form the second obstacle to simplicity:</p>
<blockquote>
<p>The costs of complex solutions are distributed across many people while the benefits of complexity tend to be concentrated in the hands of a few. These beneficiaries have, as a consequence, strong incentives to resist simplification. Much of the complexity of the U.S. tax code, for example, exists because special-interest groups secure tax breaks, including write-offs for owning a racehorse or building a race track, that benefit a small number of individuals.</p>
</blockquote>
<blockquote>
<p>After creating a labyrinth of rules, regulators and politicians often walk through the revolving door to join the companies they formerly supervised. In the private sector they can guide their new employers through the maze of regulations they themselves helped build. And the revolving door between government and business is likely to spin faster as regulatory complexity increases. A recent study found that the number of former regulators hired by financial service firms increased by 55 percent between 2001 and 2013.</p>
</blockquote>
<p>Finally, its complexity itself.</p>
<blockquote>
<p>The third obstacle to simplicity is what we call the “myth of requisite complexity,” the mistaken belief that complex problems demand complicated solutions. There are, naturally, situations when complicated solutions are appropriate—recall the detailed checklists used by pilots and surgeons. But detailed rules and regulations aren't the only possible way to deal with complexity. The U.S. Congress responded to the 2008 financial crisis by drafting tens of thousands of pages of detailed regulations covering financial service organizations.</p>
</blockquote>
<p>Whether detailed regulation wins over simple rules is an unfortunately seldom investigated empirical question. The few that exist so far, seem to point against more rules:</p>
<blockquote>
<p>Consider the results of a recent study that analyzed legal systems around the world. The authors compared how well the judicial systems in 109 countries resolved two of the most common legal disputes—evicting a tenant who stops paying rent and chasing down payment on a bounced check. Their study revealed enormous variation across countries in terms of the number of rules limiting a judge’s discretion, with more rule-laden systems taking an order of magnitude longer to mete out justice. In the United States, a landlord could evict a delinquent tenant in under two months, a process that took eighteen months in Austria. The authors measured justice along several dimensions, including whether citizens considered their legal systems impartial, free from corruption, affordable, and consistent. The number of rules was negatively correlated with all measures of justice. Any way the authors cut the data, the result was the same—more rules, less justice.</p>
</blockquote>
<p>Sometimes forgoing rules gives a firm a competitive advantage.</p>
<blockquote>
<p>After studying their human resources policies, executives at Netflix determined that 97 percent of their employees were trustworthy. Nearly all of the company’s time writing, monitoring, and enforcing detailed personnel policies was directed at the remaining 3 percent. Rather than continue to produce binders of detailed regulations, Netflix executives concentrated on not hiring people who would cause problems, and removing them quickly when hiring mistakes were made. This change allowed the company to replace thick manuals with simple rules. The company’s policy for expenses, travel, gifts, and conducting personal business at work, for example, was reduced to four rules: (1) expense what you would not otherwise spend, (2) travel as if it were your own money, (3) disclose nontrivial gifts from vendors, and (4) do personal stuff at work when it is inefficient not to.</p>
</blockquote>
<h2 id="examples">Examples</h2>
<h3 id="dating">Dating</h3>
<p>Authors included a chapter on the online dating experience of Harry, a recent architecture graduate, to illustrate the process of creating his own rules. It's difficult enough to be noticed and be attractive online, but doing it aside a busy career highlights the value of a systematic approach. After an initial frustration with the slow progress and poor dates, he went over every aspect of the process, identifying bottlenecks in his own online image to the way he messaged people.</p>
<p>While he first considered his profile and its picture to be in need of refreshing, the one-off task meant it wouldn't be a good candidate for simple rules. Further analysis led him to filtering potential mates and contacting them — two activities that he'd be doing a lot and where he could improve the most. He had found, for example, that only 1 in 6 women responded to his introductions. Out of women that had responded, half led to the exchange of phone numbers, and ⅔ of those to a date. Dates were fifty-fifty, leading to a total of one out of 36 matches being an enjoyable date.</p>
<p>Based on his earlier messages he come up with the first rule:</p>
<blockquote>
<p>Send feelers before essays. Sending short messages to establish interest, then following up with gradually longer ones, was a better course of action.</p>
</blockquote>
<p>Second, he looked at dates:</p>
<blockquote>
<p>Harry noted a common pattern to these dates. It seemed like a good idea when they agreed to go out, but on the way to the restaurant he realized he was not particularly excited about dinner. These dates were often pleasant enough, but they represented a significant opportunity cost, in Harry’s view, because his work schedule meant he could only go out a few nights a week at most. This observation based on experience led to Harry’s second rule: “Only pursue her if you would like to see her tonight.” By making theoretical opportunities immediate, this rule helped Harry avoid approaching potential dates where they had little in common.</p>
</blockquote>
<p>Then, at his selection process:</p>
<blockquote>
<p>Another source of disappointing dates occurred when someone posted pictures and then didn’t look anything like their photos when they arrived. Harry was not overly hung up on physical appearance, but he was bothered by the attempt at deception. He turned to his friend Will, who had been dating online for a while, for advice. Will explained that he had encountered the same problem early in his online dating career, but eventually learned a few tactics to detect misleading photographs. Will’s tips centered around photographic variety: if there was none, especially if all the pictures were taken from the same angle, it raised a red flag. Will also cautioned against “beautiful outliers,” which occurred when one picture was much more fetching than others. Harry added a third rule to “avoid photographic red flags”</p>
</blockquote>
<p>Harry also looked into testing his initial messages based on the results of <a href="https://www.okcupid.com">OkCupid</a>'s article on <a href="https://www.gwern.net/docs/psychology/okcupid/exactlywhattosayinafirstmessage.html">"Exactly What To Say In A First Message"</a> and ended up with a simple "Ask her how it's going".</p>
<h3 id="charisma">Charisma</h3>
<p>There was also a story of a mechanical engineer coming up with his own rules on how to be a better communicator, reading among others <a href="https://foxcabane.com/book/">"The Charisma Myth: How Anyone Can Master the Art and Science of Personal Magnetism" by Olivia Fox Cabane</a>, and came up with the following personal rules:</p>
<ol>
<li><q>Imagine the person you are talking to is the sympathetic star in a film you are watching.</q></li>
<li><q>Carry yourself like a king — calm, comfortable, and without excessive nodding, “uh-huh”-ing, and fidgeting.</q></li>
<li><q>Make and maintain soft eye contact, which means relaxing your eyes and face when you look at someone. By maintaining soft eye contact, Daniel learned, he could focus on what the other person was saying and build a stronger connection as they spoke.</q></li>
</ol>
<h2 id="appendix">Appendix</h2>
<p>Both authors have given talks about the book. Out of the two below, I recommend Donald Sull's.</p>
<ul>
<li><a href="https://www.youtube.com/watch?v=BgD3V-IyVT4">Simple Rules -- How to Thrive in a Complex World (2015) - Donald Sull - Talks at Google</a></li>
<li><a href="https://www.youtube.com/watch?v=c-pujS_7OHM">Simple Rules for a Complex World (2015) - Kathleen Eisenhardt - Stanford eCorner</a></li>
</ul>https://m811.com/articles/notes-on-zero-downtime-deploys-for-rails-talk/Notes on «Zero Downtime Deploys for Rails» talk2019-09-18T01:02:00.000Z2019-09-18T01:02:00.000Z<p>Ruby on Rails-focused, but equally applicable to webapps in JavaScript/Node.js and beyond. Describes techniques to achieve deployments that do not require downtime, even for those that include database schema changes. While downtime for database changes in my view can be excused, the HTML-compatibility sections are very applicable to JavaScript heavy sites that may run for extended periods in the client's browser. Some technical suggested solutions are irrelevant in 2018, but overall timeless.</p>
<h2 id="database-compatibility">Database Compatibility</h2>
<p>Consider removing a model attribute and its related database column. Along with the necessary code changes, you create a database migration:</p>
<pre><code class="hljs language-ruby"><span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">ApparentlyHarmlessMigration</span> < ActiveRecord::Migration</span>
def <span class="hljs-keyword">self</span>.up
remove_column <span class="hljs-symbol">:users</span>, <span class="hljs-symbol">:notes</span>
<span class="hljs-keyword">end</span>
<span class="hljs-keyword">end</span>
</code></pre>
<p>Upon deploying, however, you're likely to see your existing webapp instances raise database errors:</p>
<pre><code class="">PGError: ERROR: column "notes" does not exist
</code></pre>
<p><a href="https://guides.rubyonrails.org/active_record_basics.html">Ruby on Rails' ActiveRecord</a>, as any other ORM, will cache columns for performance in the production environment. App instances servicing requests that started before your deployment therefore still reference the removed <code class="">notes</code> column. Note this is likely the case even if you're not using a full ORM and are passing plain objects to a query library (akin to <code class="">query.table("users").insert(user)</code>) as those objects probably still refer to non-existent columns.</p>
<p>Pedro proposes approaching this problem through <strong>hot compatibility</strong> --- ensuring every two consecutive deployments be compatible with each other and able to run in parallel. For example, removing a column requires 2 deployments:</p>
<ol start="0">
<li>
<p>Write to <code class="">notes</code>.</p>
</li>
<li>
<p>Stop referencing <code class="">notes</code> while leaving it in the database:</p>
<pre><code class="hljs language-ruby"><span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">User</span></span>
def <span class="hljs-keyword">self</span>.columns
<span class="hljs-keyword">super</span>.reject {<span class="hljs-params">|column|</span> column.name == <span class="hljs-string">"notes"</span> }
<span class="hljs-keyword">end</span>
<span class="hljs-keyword">end</span>
</code></pre>
</li>
<li>
<p>Remove <code class="">notes</code> from the database.</p>
</li>
</ol>
<p>For renaming columns, Pedro suggests temporarily writing to both columns, requiring a total of 3 deployments:</p>
<ol start="0">
<li>Read and write to <code class="">notes</code>.</li>
<li>Add column <code class="">remarks</code>. Read from <code class="">notes</code>, write to <code class="">notes</code> <em>and</em> <code class="">remarks</code>.</li>
<li>Populate <code class="">remarks</code> where needed; read and write only to <code class="">remarks</code>.</li>
<li>Remove <code class="">notes</code>.</li>
</ol>
<h2 id="html-compatibility">HTML Compatibility</h2>
<p>Beyond model and database mismatches, you're likely to see problems with HTML forms and API requests that were submitted from a page rendered by a previous version of the app. Take a form on the signin page with a renamed field for example:</p>
<pre><code class="hljs language-patch"> <form method="post" action="/session">
<span class="hljs-deletion">- <input name="username"></span>
<span class="hljs-addition">+ <input name="email"></span>
<button>Sign in</button>
</form>
</code></pre>
<p>In a fully server-side rendered app perhaps not that problematic --- submitting it would merely inform the visitor of not having filled their email field while presenting them with an updated form<sup id="fnref-1"><a href="https://m811.com/articles/notes-on-zero-downtime-deploys-for-rails-talk/#fn-1" class="footnote-ref">1</a></sup>. A JavaScript-heavy signin form (or a single-page app), on the other, is bound to just break entirely. In both server- and client-side approaches there's a high risk of losing the person's submitted information and that's a cardinal sin of user experience.</p>
<p>A solution is to be backwards compatible with old requests and migrate parameters in the controller:</p>
<pre><code class="hljs language-ruby"><span class="hljs-class"><span class="hljs-keyword">class</span> <span class="hljs-title">AuthController</span> < ApplicationController</span>
<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">filtered_params</span></span>
params.dup.tap <span class="hljs-keyword">do</span> <span class="hljs-params">|params|</span>
params.merge!(<span class="hljs-symbol">email:</span> params.delete(<span class="hljs-symbol">:username</span>))
<span class="hljs-keyword">end</span>
<span class="hljs-keyword">end</span>
<span class="hljs-keyword">end</span>
</code></pre>
<p>There's also a potential problem with <strong>page assets</strong>. If you're <strong>removing or renaming CSS stylesheets or JavaScript files</strong>, it's possible a person loading your page in the middle of a migration will reference assets you've removed in the latest deployment. Functional and visual anomalies can also happen if you change stylesheets or JavaScripts in ways that are incompatible with the HTML of the previous deployment. Pedro proposed <strong>versioning assets</strong> and <strong>keeping older assets around</strong> for some time.</p>
<h2 id="migration-strategies">Migration Strategies</h2>
<p>Once you get your app's consecutive versions compatible, it's important to ensure your deployment process doesn't prevent existing app instances from servicing requests. It's easy to accidentally lock entire database tables through migrations.</p>
<p>For PostgreSQL, the locking implications are:</p>
<table><thead><tr><th>Operation</th><th>Performance</th></tr></thead><tbody><tr><td><code class="">ADD COLUMN</code></td><td>O(n) lock if you set a default value (writes to <em>every</em> row). O(1) otherwise.</td></tr><tr><td><code class="">ALTER COLUMN</code></td><td>O(N) lock.</td></tr><tr><td><code class="">REMOVE COLUMN</code></td><td>O(1)</td></tr><tr><td><code class="">CREATE INDEX</code></td><td>O(N) lock unless you create via <code class="">CREATE INDEX CONCURRENTLY</code>.</td></tr><tr><td><code class="">DELETE INDEX</code></td><td>O(1)</td></tr></tbody></table>
<p>To minimize the time of holding a lock on the entire table when adding a column with a default value, split the operation to two parts. One that adds a column with no default value and the other that sets it:</p>
<pre><code class="hljs language-sql"><span class="hljs-keyword">ALTER</span> <span class="hljs-keyword">TABLE</span> users <span class="hljs-keyword">ADD</span> <span class="hljs-keyword">COLUMN</span> notes TEXT;
<span class="hljs-keyword">ALTER</span> <span class="hljs-keyword">TABLE</span> users <span class="hljs-keyword">ALTER</span> <span class="hljs-keyword">COLUMN</span> notes <span class="hljs-keyword">SET</span> <span class="hljs-keyword">DEFAULT</span> "";
</code></pre>
<p>Note that this will leave existing rows and the column itself nullable. You should probably do batched updates (e.g. setting a few thousand rows at a time) at some point to fill in the missing values and then alter the column to <code class="">NOT NULL</code>.</p>
<p>PostgreSQL 11 is said to <a href="https://www.depesz.com/2018/04/04/waiting-for-postgresql-11-fast-alter-table-add-column-with-a-non-null-default/">provide fast column additions even with default values</a>.</p>
<h2 id="server">Server</h2>
<p>For being able to do zero downtime deployments, you'll also need a web server that can start up new instances and gracefully shut down old ones while leaving existing running requests to finish on their own.</p>
<p>This can be handled "above the stack" by an external load balancer (Nginx, HAProxy or Heroku's platform) in front of your Ruby webserver that optionally coordinates with your app when to stop routing requests. Alternatively, you can go with <a href="https://bogomips.org/unicorn/">Unicorn</a> that can also handle graceful restarts.</p>
<div class="footnotes">
<hr />
<ol>
<li id="fn-1">Although imagine the resulting frustration if that message instructed to click the browser's <em>back</em> button and try again instead, only to have the error persist.<a href="https://m811.com/articles/notes-on-zero-downtime-deploys-for-rails-talk/#fnref-1" class="footnote-backref">↩</a></li>
</ol>
</div>https://m811.com/articles/notes-on-study-less-study-smart-talk/Notes on «Study Less, Study Smart» talk2019-09-18T01:02:00.000Z2019-09-18T01:02:00.000Z<big>
<p>Marty Lobdell, after having been a professor of psychology for 40 years, summarizes better methods of studying and common mistakes.</p>
</big>
<h2 id="study-in-chunks">Study in Chunks</h2>
<p>Loving to teach by anecdote, Marty begins with a story of an ex-girlfriend that was falling behind in college. To better prepare for exams, she set herself a goal to study non-stop for 6 hours five evenings a week. However, she ended with worse grades than she started with. Turns out, a study of learning effectiveness found that <strong>on average a person can effectively study for approximately 25–30 minutes at a time</strong>, after which they begin to tune out. Out of the 6 hours she set aside, 5 and a half went at best to waste, at worst kindling a hatred for the subject and its textbook.</p>
<p><em>The moment you start to slide, you're shoveling against the tide.</em> Fortunately it doesn't take much to recharge your batteries — <strong>do something fun for 5 minutes</strong>. Upon returning, your performance is near its start. At the end of the study period and before reviewing the material, give yourself a big treat. Marty throws in another anecdote of his classmates studying after having spent an evening in the bar and anchoring their memories (<a href="https://en.wikipedia.org/wiki/State-dependent_memory">state-dependent memory</a>) with their intoxication. Not a good idea due to have to show up to the exam drunk.</p>
<p>To those complaining about lack of time to study, look for breaks at work or at home. 15–20 minutes could be sufficient if applied efficiently. If you plan your day right, you can have plenty of study possibilities. And with general positive reinforcement, you'll end up extending the study muscle by eventually doubling or tripling the time before you need a break.</p>
<h2 id="study-area">Study Area</h2>
<p>Marty goes over problems with various locations for study. The bedroom, used primarily for sleeping, has conditioned people to feel tired. A kitchen will tempt with food. Living rooms, when shared with others, offer or demand distraction, unless you're by yourself. Background music is fine, but if you're singing along to a song, you're not studying.</p>
<p>Hawaii University's students say the biggest study problem is getting into it. Every dorm room there came with a particular desk lamp. Professors then proposed that lamp be used only for studying, not for regular lighting or during breaks. Doing so for some time will then condition you to automatically switch to study mode when its turned on. With state-dependent memory being mentioned, I wonder how that ties into <a href="https://en.wikipedia.org/wiki/Context-dependent_memory">context-dependent memory</a>. Students who did so were supposedly one grade point higher than the control group.</p>
<p>Marty urges people to try both chunking time and allocating a study area. After all, <q>if it doesn't change your behavior, you haven't learned it.</q></p>
<h2 id="study-method">Study Method</h2>
<blockquote>
<p>The more active you are in your learning, the more effective.</p>
</blockquote>
<p>We were all taught through rote memorization in elementary schools. While mechanic repetition may work for some, it isn't the optimal method for the majority. First, you have to <strong>distinguish whether you're learning a fact — a discrete piece of information — or a concept</strong>. Fact would be "Sigmund Freud is the father of psychoanalysis"; "What psychoanalysis is" a concept. <q>Concepts, once grasped, will stay with you for the entire lifetime.</q> Facts you can always look up. The more effort you expend thinking about something and the more meaning found in it, the better it's remembered. Right after a lecture, Marty suggests expanding your short-hand notes. If you do it hours later, you may have forgotten what you wrote. <q>Can you put the concept in your own words? If you can't, you don't understand it.</q></p>
<p><strong>People are incredible at confusing recollection with recognition.</strong> As an example, Marty brings up highlighting literature. Assuming only important parts are highlighted, a crucial mistake when reviewing highlights later is to confuse recognition of information as confirmation of its knowledge, resulting eventually in knowing exactly none of the important facts. Trying to restate it in your own words helps. Teaching someone else, a friend, a parent or even an imaginary person, is a great way to recapitulate your learning.</p>
<p>People also un-do good studying by not sleeping well. It's said memory consolidation is dependent on REM sleep, which you may not get enough of when sleeping less than ~8h per night.</p>
<h2 id="reading-method">Reading Method</h2>
<p>Marty suggests working with textbooks through the <a href="https://en.wikipedia.org/wiki/SQ3R">SQ3R</a> method:</p>
<ul>
<li><strong>S</strong>urvey — Start by browsing through the chapter, spotting interesting things.</li>
<li><strong>Q</strong>uestion — While browsing, formulate questions about unfamiliar and interesting things.</li>
<li><strong>R</strong>ead — Later, when reading, questions guide you to seek answers.</li>
<li><strong>R</strong>ecite</li>
<li><strong>R</strong>eview</li>
</ul>
<h2 id="mnemonics">Mnemonics</h2>
<p>In place of rote memorization, Marty proposes mnemonics as a great method to remember facts. He lists various fun examples, from acronyms to fun children's sayings, as ways to remember calorie contents, to <a href="https://en.wikipedia.org/wiki/ROYGBIV">acronyms for rainbow colors</a> and anatomy.</p>
<h2 id="book">Book</h2>
<p>"Study Less, Study Smart" also available as a 37-page book from <a href="https://www.amazon.com/Study-Less-Smart-Marty-Lobdell-ebook/dp/B00URRJL24">Amazon</a> (<a href="https://www.youtube.com/watch?v=y-sN6zywKIE">introduced by Marty on YouTube</a>).</p>https://m811.com/articles/notes-on-deconstructing-the-database-talk/Notes on «Deconstructing the Database» talk2019-09-18T01:02:00.000Z2019-09-18T01:02:00.000Z<h2 id="problem">Problem</h2>
<p>Common to Rich Hickey, he starts with a proposition that the biggest problem in programming is complexity. According to Ben Moseley and Peter Marks's "<a href="http://curtclifton.net/papers/MoseleyMarks06a.pdf">Out of the Tar Pit</a>", the major contributors to complexity are state and [flow of] control, and the solutions functional programming and Edgar F. Codd's <a href="https://en.wikipedia.org/wiki/Relational_model">relational model</a> of data. Rich accepts the relational model (declarative programming) being superior to imperative means of data processing (e.g. object-oriented or procedural programming), but carries on critiquing its model and common client-server implementations.</p>
<h3 id="lack-of-basis">Lack of Basis</h3>
<p>Due to relational databases not modelling nor storing time explicitly, computations resulting from their contents aren't repeatable. Under a dataset that's in constant change, a response to a query today may differ from the response of tomorrow. This leads to coupling questions with reporting, where the question of what to display is combined with the query to fetch the data --- the justified fear of querying at separate times leading to inconsistent views of the data.</p>
<h3 id="consistency-and-scale">Consistency and Scale</h3>
<p>Rich briefly alludes to limitations with monolithic architectures and asks, how many value propositions of relational databases can we retain while gaining from distributed systems? People are trading in consistency for performance, gaining unnecessary complexity, and that's not a good trade-off.</p>
<h3 id="flexibility">Flexibility</h3>
<p>Some pieces of data are supposedly hard to represent in a relational model, like irregular and hierarchical data, and multi-valued attributes. Inflexibility in the database will reflect in the app as unnecessary complexity due to having to work-around those limitations.</p>
<p>I think it's valuable here to distinguish between the relational model as it stood originally, in it's ideal normalized form, and today's implementations which are more flexible in types and forms than Rich talks about. Hierarchical and irregular data (presumably data without a schema), however, remain to the most extent clunky, although query recursion and indexing has progressed to where practically they may not present problems. Handling of time --- the <em>basis</em> he's talking about --- remains non-existent beyond the scope of a single database transaction.</p>
<h3 id="perception-and-reactions">Perception and Reactions</h3>
<p>With perception being querying and reaction being notifications or change-events, he brings up the problem of basis again --- the reaction to change happens at a later time when the dataset may have already changed again.</p>
<p>I liked his jab at key-value stores: It's the indexing that provides leverage in databases. Without that, they don't differ from other primitive stores like plain files or file systems and shouldn't be called databases.</p>
<h2 id="approach">Approach</h2>
<p>He proposes a new distributed and fact-driven database named <a href="https://www.datomic.com">Datomic</a>.</p>
<h3 id="information-model">Information Model</h3>
<p>Replace structure (rows and columns) with a collection of single facts. Similarly to <a href="https://en.wikipedia.org/wiki/Resource_Description_Framework">RDF</a>, with subject, predicate and object, with the addition of a temporal component. Temporal component isn't limited to time, but can also include the transaction details, author and other auditing information.</p>
<p>This reminds me of the problem of defining the semantics of <code class="">created_at</code> columns in today's databases. While they're often meant to refer to the time the row was created, I've argued they should be semantically utilized and store the meaningful creation time. That may not coincide with the row creation time. For example, a note created offline should have as its creation time the real time it was created, not the time it was received by the server. Later in the talk, Rich brings up the lack of coordination requirement when querying as a benefit, drawing parallels to the real world's distributive nature. Here, however, he seems to glance over it. How does Datomic's convenient temporal modelling play with the real world we're modelling that accrues facts in a distribute nature? Can you nest them, with the outer one referring to the database's [eventual] knowledge and the inner one to the world? I know we're treading into "eventual consistency" here, but it definitely reduces the value proposition if we're then back to manually propagating time in <code class="">JOIN</code>s.</p>
<h3 id="state">State</h3>
<p>To solve the problem of novelty, he proposes never to change the past, only accrue new facts through assertionss and deletions through retractions of facts. Every other action will reduce down to those two. Similarly to append-only transaction logs, only with their temporal aspect in the foreground, permitting referencing any state between now and beginning.</p>
<h2 id="implementation">Implementation</h2>
<p>Proposes splitting process (mutation) from perception/reaction (reading), permitting independent scaling. Storage locality would be irrelevant, as the difference between in-memory and on-disk is vastly larger than the difference between local-memory and remote-memory. Leads to treating storage as a key-value black box to store immutable facts, requiring only consistent-reads from it. In addition to fact storage, requires a bit of mutable storage for the root of the tree. The tree itself can be a structurally shared immutable tree:</p>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-deconstructing-the-database-talk/structural-sharing.png"><img src="https://m811.com/articles/notes-on-deconstructing-the-database-talk/structural-sharing.png" alt="Structural sharing graph" /></a></p>
</figure>
<h3 id="transaction-and-indexing">Transaction and Indexing</h3>
<p>Indexing seems to be a mix between stored indices and in-memory indices, occasionally dumping the latter out and merging.</p>
<h3 id="perception">Perception</h3>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-deconstructing-the-database-talk/perception.png"><img src="https://m811.com/articles/notes-on-deconstructing-the-database-talk/perception.png" alt="Perception graph" /></a></p>
</figure>
<p>Perception only requires access to the live index and storage. No other components necessary, incl. no coordination necessary or transaction processing system.</p>
<h3 id="components">Components</h3>
<figure class="slide">
<p><a href="https://m811.com/articles/notes-on-deconstructing-the-database-talk/architecture.png"><img src="https://m811.com/articles/notes-on-deconstructing-the-database-talk/architecture.png" alt="Datomic architecture graph" /></a></p>
<figcaption>
Datomic architecture graph
</figcaption>
</figure>
<p><strong>Writing</strong>: For storing new facts, an app will communicate with the transactor, which will both store the fact and transmit novelty to peers for a live index. The transactor accepts either new <strong>assertions</strong>, <strong>retractions</strong> or <strong>transaction functions</strong> which will be given the current database and can return either more transaction functions or facts.</p>
<p><strong>Reading</strong>: Querying will bypass the transactor and go to storage directly, with optional caching. The query engine can live anywhere, as long as it has access to the storage. The index will be a combination of the durable index from storage and the live index with changes <em>since</em> the last time the storage index was updated.</p>
<h2 id="elsewhere">Elsewhere</h2>
<ul>
<li><a href="https://news.ycombinator.com/item?id=4444793">Hacker News Discussion Thread</a></li>
<li><a href="https://www.reddit.com/r/datomic/">Datomic Reddit</a></li>
</ul>https://m811.com/articles/notes-on-the-diversity-delusion-book/Notes on «The Diversity Delusion» book2019-09-18T01:02:00.000Z2019-09-18T01:02:00.000Z<big>
<p>In <strong>The Diversity Delusion</strong>, Heather Mac Donald describes, analyzes and critiques the race, gender and identity politics of colleges and society of the recent years. She puts forward the argument that claims of minority discrimination and oppression are baseless, and any inequality or disproportional representation of minorities can be explained by objective differences of individuals. She also critiques solutions that attempt to achieve equality of outcome by selectively lowering standards and by further discrimination, i.e <a href="https://en.wikipedia.org/wiki/Affirmative_action">affirmative action</a>. Both would be not only unjust, but counter-productive to achieving said equality.</p>
</big>
<p>Overall an easy read with majority of the pages devoted to stories of diversity initiatives gone wrong in modern United States colleges. Her argument relies mostly on absurdity of the examples and is therefore unlikely to convince supporters of those initiatives. For that, I'd recommend Thomas Sowell's <a href="https://en.wikipedia.org/wiki/Affirmative_Action_Around_the_World">Affirmative Action Around the World (2004)</a>. For an shorter introduction to the book and its stories, see <a href="https://youtu.be/V-DbLiTWY58">Dave Rubin's interview with Heather Mac Donald</a>.</p>
<h2 id="problem">Problem</h2>
<p>The past few years have seen strong erosion of classical Enlightenment values, of fairness, <a href="https://en.wikipedia.org/wiki/Meritocracy">meritocracy</a>, reason and due process. No longer are students viewing themselves as individuals, but as groups defined by race, sex and sexual preference, and based on these properties they claim to be discriminated against. Evidence for any actual discrimination is weak, to say the least, and any there is, is more based on ideology than reason. While the revolt started in progressive colleges, it has since moved on to affect businesses, governments and society under the category of "identity politics".</p>
<p>Heather sets the stage by describing a few eerie incidents.</p>
<blockquote>
<p>A <a href="https://www.youtube.com/watch?v=9IEFD_JVYd0">notorious video</a> of a black female student at Yale screaming and cursing at her college master in November 2015 is a chilling portrait of self-engrossed, bathos-filled entitlement that has never been corrected by truth, much less restrained by manners: “Be quiet!” she shrieks at the frozen administrator. “Why the fuck did you accept the [master] position, who the fuck hired you?!” she continues at full, self-righteous cry. “You should not sleep at night! You are disgusting!”
The master’s wife, child psychologist Erika Christakis, had recently suggested in an email that the Yale multiculturalism bureaucracy did not need to oversee Halloween costumes. […] A hundred or so mostly minority students then mobbed her husband, Nicholas Christakis, a renowned physician and sociologist, for an hours-long abuse session in the college quad that included the “Be quiet!” shriek, among equally horrifying displays of rudeness. “You are disgusting!” screamed another student. “I want your job to be taken from you.</p>
</blockquote>
<p>Another, as covered by The New York Times in <a href="https://www.nytimes.com/2017/03/13/opinion/understanding-the-angry-mob-that-gave-me-a-concussion.html">Understanding the Angry Mob at Middlebury That Gave Me a Concussion</a>:</p>
<blockquote>
<p>In March 2017, a mob of Middlebury College students assaulted a professor, giving her a concussion and whiplash, following their successful effort to prevent social scientist Charles Murray from speaking to a live audience by shouting, pounding on walls, and activating fire alarms. Murray just missed being knocked down and beaten himself. […] The professors blamed the Middlebury administration for the violence, since its decision to allow Murray to lecture constituted a “threat” to students.</p>
</blockquote>
<p>Or the Evergreen State College controversy where only the white faculty and students were ordered to cancel their courses for the day and stay at home to show solidarity with the minority students' struggles. Biology professor <a href="https://bretweinstein.net/">Bret Weinstein</a> refused to obey the edict and was met with a mob of students:</p>
<blockquote>
<p>In May 2017 students from Evergreen State College in Washington state stormed into a class taught by biology professor Bret Weinstein and began cursing and hurling racial epithets. “Fuck you, you piece of shit,” screamed one student. “Get the fuck out of here,” screamed another. […] Weinstein told the mob that he did not believe that science professors at Evergreen were “targeting” students of color, contrary to the premises of a newly announced equity initiative. “Fuck what you have to say,” a student responded. “This is not a discussion.” Evergreen’s president, after being subjected to a similar expletive-filled mob tirade, expressed his “gratitude for the [students’] passion and courage.”</p>
</blockquote>
<p>For clips of the sort of cult-like brainwashing that may have led to the entire incident, see the video of <a href="https://youtu.be/FH2WeWgcSMk">Bret Weinstein and Heather Heying on the Evergreen Equity Council</a>.</p>
<p>The author had experienced a similar response herself:</p>
<blockquote>
<p>The Rose Institute for State and Local Government at Claremont McKenna had invited me to meet with students and to give a talk in April about my book The War on Cops. Several calls went out on Facebook to “shut down” this “notorious white supremacist fascist Heather Mac Donald.” […] The event organizers notified me a day before the speech that a protest was planned and that they were considering changing the venue from CMC’s Athenaeum to one with fewer glass windows and easier egress. When I arrived on campus, I was shuttled to what was, in effect, a safe house: a guest suite for campus visitors, with blinds drawn. I could hear the growing crowds chanting and drumming, but I could not see the auditorium that the protesters were surrounding. […] Shortly before 6 PM, I was fetched by an administrator and a few police officers to take an out-of-the-way elevator into the Athenaeum. The massive hall, where I was supposed to meet with students for dinner before my talk, was empty—the mob, by then numbering close to two hundred, had succeeded in preventing anyone from entering.</p>
</blockquote>
<p>You can hear her talk about the experience in <a href="https://youtu.be/V-DbLiTWY58?t=62">Dave Rubin's interview with Heather Mac Donald</a> or watch a <a href="https://www.youtube.com/watch?v=S8yck_4ixgI">video of the protesters</a> chanting "shut it down" and later "fuck the police". Heather notes the irony:</p>
<blockquote>
<p>The only people bullying and intimidating others were the students themselves, but that fact does not penetrate the upside-down world of campus victimology.</p>
</blockquote>
<p>University responses to the incidents above have unfortunately all been to succumb to the demands of the mobs, not standing up for reason, civil discourse, freedom of speech nor their unjustly accused professors.</p>
<blockquote>
<p>We are cultivating students who lack all understanding of the principles of the American Founding. The mark of any civilization is its commitment to reason and discourse. The great accomplishment of the European Enlightenment was to require all forms of authority to justify themselves through rational argument, rather than through coercion or an unadorned appeal to tradition. The resort to brute force in the face of disagreement is particularly disturbing in a university, which should provide a model of civil discourse.</p>
</blockquote>
<p>What caused the movement, is less clear. One explanation is the snowflake theory — helicopter parents of <a href="https://en.wikipedia.org/wiki/Generation_Z">Generation Z</a> have done a disservice to their children by overprotecting them and have now created adults ill-equipped to handle foreign ideas or interpersonal conflict. That's a position Greg Lukianoff and Jonathan Haidt made in <a href="https://www.theatlantic.com/magazine/archive/2015/09/the-coddling-of-the-american-mind/399356/">The Coddling of the American Mind</a>. If the psychological harm claim were true, they propose cognitive behavioral therapy as a means to handle that. Heather Mac Donald, however, doesn't agree with the snowflake theory on the basis that if child-rearing were the problem, it should show in equal proportion in heterosexual white males, who presuming too had the same risk-averse parents. Instead, there could be an ideological reason.</p>
<blockquote>
<p>At its center is a worldview that sees Western culture as endemically racist and sexist. The overriding goal of the educational establishment is to teach young people within the ever-growing list of official victim classifications to view themselves as existentially oppressed. One outcome of that teaching is the forceful silencing of contrarian speech.</p>
</blockquote>
<p>She continues to warn:</p>
<blockquote>
<p>Many observers dismiss such ignorant tantrums as a phase that will end once the “snowflakes” encounter the real world. But the graduates of the academic victimology complex are remaking the world in their image.</p>
</blockquote>
<p>Take the firing of Google software developer James Damore in 2017 for writing an internal document <a href="https://assets.documentcloud.org/documents/3914586/Googles-Ideological-Echo-Chamber.pdf">"Google’s Ideological Echo Chamber"</a>, for example. The document lists scientific literature in gender differences in response to why there might be fewer women in Google than elsewhere. Damore also highlights discriminatory hiring in favor of women and minorities at Google. <a href="https://www.youtube.com/watch?v=SEDuVF7kiPU">Jordan Peterson in an interview with James Damore</a> went through the memo line by line for those interested in the details.</p>
<p>Damore went to the National Labor Relations Board to appeal the termination. Unfortunately, however, the censoring of politically incorrect statements, regardless of truth, seem to be creeping beyond academia and business to government bureaucrats:</p>
<blockquote>
<p>Damore’s statements about “purported biological differences between men and women” were “discriminatory and constituted sexual harassment,” declared NLRB counsel Jayme Sophir. Sophir sneers that Damore tried to cloak his comments with “‘scientific’ references and analysis.” She makes no effort to determine whether that science met traditional research standards, which it does. If it contradicts feminist ideology, it must be both wrong and suppressed. Sophir notes that some of Google’s employees had complained that Damore’s memo made them feel “unsafe at work.” Thus does bathos-filled academic victimology get bootstrapped into further assaults on rational inquiry outside the academy.</p>
</blockquote>
<h2 id="race">Race</h2>
<p>In 1996, California approved the <a href="https://en.wikipedia.org/wiki/1996_California_Proposition_209">California Civil Rights Initiative</a> with the support of 5.26M citizens (forming the majority of 54.55%) to ban discrimination by race or gender in government positions and education. Upon its approval, while few politicians publicly denounced it, plenty expected it to be being reversed soon and thereby ignored it. It fell to the <a href="https://www.nytimes.com/1997/04/09/us/federal-appeals-court-upholds-california-s-ban-on-preferences.html">9th Circuit Court of Appeals a year later to uphold the law</a>.</p>
<blockquote>
<p>From then on, state and federal judges would show an admirable respect both for voter intent and for the plain meaning of the state’s new constitutional amendment. Not so for California’s bureaucrats and pols. Many chose passive resistance or tried to hide noncompliance under Orwellian name changes: San Jose’s affirmative action bureaucracy rechristened itself the “Office of Equality Assurance,” for instance.</p>
</blockquote>
<p>Without the effort of a public interest law firm, though, some of the biggest governmental agencies would have continued their discriminatory actions. At one point in 2003 the state assembly even attempted to sidestep the ban by adopting the United Nations' <a href="https://en.wikipedia.org/wiki/International_Convention_on_the_Elimination_of_All_Forms_of_Racial_Discrimination#Prohibition_of_incitement">International Convention on the Elimination of All Forms of Racial Discrimination</a>, whose terms suggest positive discrimination. Fortunately the California courts saw through this a year later and overruled the interpretation.</p>
<p>Heather then brings up a claim from 2004 that, if true, shows how many noncompetitive firms relied on preferential treatment:</p>
<blockquote>
<p>A propreference organization claimed in 2004 that transportation-construction contracts awarded to minority-owned businesses had dropped 50 percent since 1996 and that the percentage of women in the construction trades had declined by one third.</p>
</blockquote>
<p>Who was winning from spending taxpayers money on corporations that would not be selected based on their competence or price is beyond me.</p>
<h3 id="colleges">Colleges</h3>
<p>The fight <em>for</em> positive discrimination carried on in colleges, which were facing a crisis of lowering diversity.</p>
<blockquote>
<p>After Prop. 209’s passage, UC Berkeley, like the rest of the UC system, “went through a depression figuring out what to do,” says Robert Laird, Berkeley’s propreferences admissions director from 1993 to 1999. The system’s despair was understandable. It had relied on wildly unequal double standards to achieve its smattering of “underrepresented minorities,” especially at Berkeley and UCLA, the most competitive campuses. The median SAT score of blacks and Hispanics in Berkeley’s liberal arts programs was 250 points lower (on a 1600-point scale) than that of whites and Asians. This test-score gap was hard to miss in the classroom. Renowned Berkeley philosophy professor John Searle, who judges affirmative action “a disaster,” recounted that “they admitted people who could barely read.”</p>
</blockquote>
<p>Still, they couldn't adopt the notion of treating people fairly and were doing anything they could to disobey.</p>
<blockquote>
<p>Yet for the preference lobby, a failing diversity student is better than no diversity student at all—because the game is not about the students but about the self-image of the institution that so beneficently extends its largesse to them. Thus, when “underrepresented minorities” accepted at Berkeley dropped by half in 1998, the first year that Prop. 209 went into effect, and by nearly that much at UCLA, the university sprang into action. […] UCLA law professor Richard Sander was on a committee to discuss what could be done after 209. “The tone among many of the faculty and administrators present was not ‘How do we comply with the law in good faith?’ but ‘What is the likelihood of getting caught if we do not comply?’” he said.</p>
</blockquote>
<p>While the number of admitted "underrepresented minorities" before adjustements were made, halved, graduation rates increased. That's what you'd expect if only academically qualified students are accepted. But that wasn't what universities wanted:</p>
<blockquote>
<p>They flung themselves into their long experimentation with different admissions schemes, with one purpose: “To maintain a racially and ethnically diverse student body,” as former UC associate president Patrick Hayashi wrote in 2005. The first scheme that the university tried was to give an admissions preference to low-income students. This device backfired, however, when it yielded a wealth of Eastern European and Vietnamese admits—not the kind of “diversity” that the university had in mind. So the campuses cut their new socioeconomic preferences in half and went back to the drawing board.</p>
</blockquote>
<p>A myriad of discriminatory admittance criteria adjustments followed. Some went as far as proposing objective measurements hold no scholarly predictive value at all. Berkley University even suggested once a student passes some minimal threshold, they'd be equally qualified as any another student. Professor Jack Citrin then proposed using lottery — that would be fair and likely admit more minorities, even if further lowering academic quality.</p>
<blockquote>
<p>Citrin’s lottery proposal went nowhere, of course. Race-conscious schools are perfectly content to use objective tests of aptitude to judge Asians and whites, and even to rank black and Hispanic students within their own group. But if you suggest using objective standards to evaluate students on a common universal scale, those standards suddenly lose all their validity.</p>
</blockquote>
<p>Others have justified their critique of objective measurements by arguing that minorities suffer psychological stress during standardized testing. If that were the case, however, you'd see them perform better than predicted in school. The reality is that black and Hispanic students perform worse than their SATs predict.</p>
<h3 id="affirmative-action">Affirmative Action</h3>
<p>The <a href="https://en.wikipedia.org/wiki/Affirmative_action">affirmative action</a> of today, also known as positive discrimination, is a belief that to right past wrongs and achieve equality in society, some groups need to be preferentially treated at the cost of other groups. In contexts of college admissions or business hiring, for example, it would require considering not only competence, but race or gender, if competence alone would result in a proportionally fewer minority members admitted than in society at large.</p>
<p>Back in 2004, in <a href="http://www.adversity.net/Sander/Systemic_Analysis_FINAL.pdf">Systematic Analysis of Affirmative Action in American Law Schools</a>, <a href="https://en.wikipedia.org/wiki/Richard_Sander">Richard H. Sander</a> made a case how positive discrimination has counter-productive results:</p>
<blockquote>
<p>Because of the “mismatch” between their academic preparedness and the academic sophistication of the school that has bootstrapped them in, the preference beneficiaries learn less of what they need to pass the bar than they would in a school that matched their capabilities. Far from increasing the supply of black lawyers, affirmative action actually decreases the diversity of the bar.</p>
</blockquote>
<p>Duke University claimed their affirmative action program benefits blacks because by the senior year, the gap between whites and blacks at the time of admission (1 standard deviation) has been reduced by 50%. After further research, that improvement turned out to be illusory:</p>
<blockquote>
<p>Blacks improve their GPAs because they switch disproportionately out of more demanding science and economics majors into the humanities and soft social sciences, which grade much more liberally and require less work. If black students stayed in the sciences at the same rate as whites, there would be no convergence in GPAs. And even after their exodus from the sciences, blacks don’t improve their class standing in their four years of college. […] The Arcidiacono paper suggests that admitting aspiring minority scientists to schools where they are less prepared than their peers is counterproductive.</p>
</blockquote>
<p>Unsurprisingly, research into the empirical aspects of affirmative action at Duke resulted in ad hominem attacks on its authors and of crying hostility against black students. None seem to have addressed the contents of the study. The bureaucratic response was instead to further existing "diversity" initiatives within Duke: a black student center, a black student recruiting weekend, vice provost for faculty diversity and faculty development, associate vice provost for academic diversity and more.</p>
<blockquote>
<p>But no college administration in recent history has ever said to whining students of any race or gender: “Are you joking? We’ve kowtowed to your demands long enough, now go study!” And why should the burgeoning student services bureaucracy indulge in such honesty? It depends on just such melodramatic displays of grievance for its very existence.</p>
</blockquote>
<p>But what can you expect. John Gall's remarked on the <a href="https://en.wikipedia.org/wiki/Systemantics">persistence of bureaucracy</a> back in 1975: <q>The Newtonian Law of Systems Inertia — A system that performs a certain way will continue to operate in that way regardless of the need or of changed conditions.</q></p>
<h3 id="implicit-racism">Implicit Racism</h3>
<p>Not only are we supposedly racist, we're racist in ways we don't even consciously perceive. That's the theory of both microaggressions, implicit biases and implicit association tests (IAT). Microaggressions are subtle or unconscious verbal or non-verbal insults against marginalized groups. They can be as innocent as asking where someone is from, a question that "clearly" hints one assumes someone is foreign. Suitably, skepticism of someone's recurrent discrimination, the thesis of this book, is also a form of microaggression. Implicit biases are similar to microaggressions, but could be both positive or negative attitudes or held stereotypes. The <a href="https://en.wikipedia.org/wiki/Implicit-association_test">implicit-associtation test</a>, derived from response-time and implicit-cognition tests of earlier days, measures the automatic association strength between two concepts. Ease with which you can associate positive and negative terms with faces of different color. Should it take you longer to associate positive terms with black faces, you're a racist.</p>
<p>Validity of the implicit association test is under fire, though. It's not clear that IAT measures both implicit bias and, by extension, discriminatory behavior. Skeptics of IAT found that in a <a href="https://www.ncbi.nlm.nih.gov/pubmed/19586237">2009 meta-study of IAT</a> by its original authors Greenwald and Banaji, both positive and negative results of IAT were counted towards validity:</p>
<blockquote>
<p>If test subjects scored high on implicit bias via the IAT but demonstrated better behavior toward out-group members (such as blacks) than toward in-group members, that was a validation of the IAT on the theory that the subjects were overcompensating for their implicit bias. But studies that found a correlation between a high implicit-bias score and discriminatory behavior toward out-group members also validated the IAT. In other words: heads, I win; tails, I win.</p>
</blockquote>
<p>IAT authors have since retracted their statements that it bears predictive value for individual biases, but still predicts discrimination in aggregate. A skeptic dismisses the argument:</p>
<blockquote>
<p>If you don’t know what an instrument means on an individual level, you don’t know what it means in the aggregate, he told New York’s Singal. In fairness to Greenwald and Banaji, it is true that a cholesterol score, say, is more accurate at predicting heart attacks the larger the sample of subjects. But too much debate exists about what the IAT actually measures for much confidence about large-scale effects.</p>
</blockquote>
<h2 id="gender">Gender</h2>
<h3 id="students-sex-life">Students' Sex-Life</h3>
<blockquote>
<p>It’s a lonely job, working the phones at a college rape crisis center. Day after day, you wait for the casualties to show up from the campus rape epidemic—but few victims call. Could this mean that the crisis is overblown? No: It means, according to the campus sexual-assault industry, that the abuse of coeds is worse than anyone had ever imagined. It means that consultants and counselors need more funding to persuade student rape victims to break the silence of their suffering.</p>
</blockquote>
<p>Akin to the neuroticism around race, on-campus radical feminists are claiming women are being harassed in numbers never before experienced.</p>
<blockquote>
<p>If the one-in-five to one-in-four statistic is correct, campus rape represents a crime wave of unprecedented proportions. No crime, much less one as serious as rape, has a victimization rate remotely approaching 20 percent or 25 percent, even over many years. In 2016, the violent crime rate in Detroit, the most violent city in America, was 2,000 murders, rapes, robberies, and aggravated assaults per 100,000 inhabitants—a rate of 2 percent. The one-in-five to one-in-four statistic would mean that every year, hundreds of thousands of young women graduate who have suffered the most terrifying assault, short of murder, that a woman can experience.</p>
</blockquote>
<p>The truth seems to be both a heavy combination of fear-mongering around statistics, ideological stances of patriarchal oppression and substandard judicial process within campus, with strong victim-bias and lack of due process. The latter seemed to have resulted in retaliatory claims of rape from women either jealous or in other ways in competition with males they had had sexual relations with. The fact that colleges insisted on handling sexual assault cases on premises, without involving law enforcement, was both a signal they weren't as serious as claimed and the lack of due-process. The ones that did result in police investigations had a tendency to fall apart.</p>
<p>Another peculiarity of the sexual assault problem was the desire to unequally distribute responsibility between the involved men and women:</p>
<blockquote>
<p>Campus rape ideology holds that inebriation strips women of responsibility for their actions but preserves males’ responsibility not only for their own actions but for their partners’ as well. Thus do men again become the guardians of female well-being.</p>
</blockquote>
<p>Plans to moderate the problem led to humorous red-tape around sex as described in the <a href="http://www.alanberkowitz.com/articles/consent.pdf">Campus Safety & Student Development</a>:</p>
<blockquote>
<p>“In fact,” explains our consultant, “sexual intent can only be determined by clear and unambiguous communication about what is desired.” So much for seduction and romance; bring in the MBAs and lawyers.</p>
</blockquote>
<blockquote>
<p>“If one partner puts a condom on the other, does that signify that they are consenting to intercourse?” asked Berkowitz. Short of guiding the thus-sheathed instrumentality to port, it’s hard to imagine a clearer signal of consent.</p>
</blockquote>
<p>Why does a college want to regulate the sex-lives of grown-up people is puzzling.</p>
<blockquote>
<p>Claremont’s sexual consent rules resemble nothing so much as a multi-lawyer-drafted contract for the sale and delivery of widgets, complete with definitions, the obligations of “all” (as opposed to “both”) parties, and the preconditions for default.</p>
</blockquote>
<p>It's as if the early feminists' achievements of sexual liberation went too far in the view of modern campus feminists, and traditional social roles need to be restored. How would they reconcile that with the oppressive patriarchy narrative?</p>
<blockquote>
<p>In fact, the policy goes even further into the realm of Victorian sex roles than simply a presumption of female modesty. Females are now considered so helpless and passive that they should not even be assumed to have the strength or capacity to say “no.” “Withdrawal of Consent can be an expressed ‘no’ or can be based on an outward demonstration that conveys that an individual is hesitant, confused, uncertain, or is no longer a mutual participant,” announce Claremont’s sexocrats.
Good luck litigating that clause in a campus sex tribunal. The female can allege that the male should have known that she was “confused” because of what she didn’t do. The male will respond that he didn’t notice any particular nonactivity on her part. Resolving this evidentiary dispute would not be helped by bedside cameras—the logical next step in campus rape hysteria. Pressure sensors would be needed as well to detect asymmetries in touch.</p>
</blockquote>
<p>Equally hilarious is the <a href="https://text.npr.org/s.php?sId=321677110">handling of such conflicts</a> in the campus kangaroo courts:</p>
<blockquote>
<p>With or without cameras, adjudicating college sex in the neo-Victorian era requires a degree of prurience that should be repugnant to any self-respecting university. A campus sex investigator named Djuna Perkins described the enterprise to National Public Radio in 2014: “It will sometimes boil down to details like who turned who around, or [whether] she lifted up her body so [another student] could pull down her pants. There have been plenty of cases that I’ve done when the accused student says, ‘What do you mean? [The accuser] was moaning with pleasure. He was raising his body, clutching my back, exhibiting all signs that sounded like this was a pleasurable event.’”</p>
</blockquote>
<p>What's eerie is the view towards the current judicial process:</p>
<blockquote>
<p>Occidental College professor Caroline Heldman, a leader in the campus rape movement, asserted during a debate that campus rape cases should not be taken to criminal trial because juries are steeped in rape culture—i.e., they cannot be trusted to convict. (I was her debate opponent.) Remarkably, Heldman also argued that the preponderance-of-evidence standard for rape findings was too high. Apparently requiring that the fact-finder have a negligible 50.5-percent certainty that a rape occurred does not guarantee enough convictions. So conservatives are right to call the rape hysterics’ bluff by arguing: If you believe that this is rape, treat it as such by seeking a criminal conviction.</p>
</blockquote>
<h3 id="science">Science</h3>
<p>History has shown that affirmative action and preferential treatment programs never manage to be bounded to their initial beneficiaries. As gender and identity politics gained momentum, its leaders demanded higher representation in areas they for whatever reason on average were less skillful in or, more likely, simply lacked collective interest.</p>
<p>One such a field with gender disparities is science, or STEM (science, technology, engineering and mathematics) in general. Under the assumption that any gender difference is de facto proof of discrimination, it became the focus of diversity movements:</p>
<blockquote>
<p>A UCLA scientist reports: “All across the country the big question now in STEM is: how can we promote more women and minorities by ‘changing’ (i.e., lowering) the requirements we had previously set for graduate level study?” Mathematical problem-solving is being deemphasized in favor of more qualitative group projects; the pace of undergraduate physics education is being slowed down so that no one gets left behind.</p>
</blockquote>
<blockquote>
<p>Entry requirements for graduate education are being revised. The American Astronomical Society has recommended that PhD programs in astronomy eliminate the requirement that applicants take the Graduate Record Exam (GRE) in physics, since it has a disparate impact on females and underrepresented minorities and allegedly does not predict future research output. Harvard and other departments have complied, even though an objective test like the GRE can spotlight talent from less prestigious schools. The National Science Foundation’s Graduate Research Fellowship Program has dropped all science GREs for applicants in all fields.</p>
</blockquote>
<p>There's willingness to go as far as rejecting the entire notion of truth and aim more for "scientific self-esteem" than measurable knowledge.</p>
<blockquote>
<p>An introductory chemistry course at UC Berkeley exemplifies “culturally sensitive pedagogy.” Its creators described the course in a January 2018 webinar for STEM teachers, sponsored by the University of California’s STEM Faculty Learning Community. A primary goal of the course, according to teachers Erin Palmer and Sabriya Rosemund, is to disrupt the “racialized and gendered construct of scientific brilliance,” which defines “good science” as getting all the right answers.</p>
</blockquote>
<p>I find the erosion of meritocracy in the medical field to be especially worrisome, as we're no longer indulging mere narcissism or slowing down research, but affecting patients' well-being directly.</p>
<blockquote>
<p>Racial preferences in medical school programs are sometimes justified on the basis that minorities want doctors who “look like them.” Arguably, however, minority patients with serious illnesses want the same thing as anyone else: subject mastery.</p>
</blockquote>https://m811.com/articles/comments-on-xroad-protocol-for-rest/Comments on X-Road Protocol for REST2018-10-19T03:12:00.000Z2018-10-19T03:12:00.000Z<p>Comments below refer to the <a href="https://www.niis.org/s/X-Road_Message_Protocol_for_REST-v010.pdf">X-Road Message Protocol for REST draft v0.1</a>, as <a href="https://www.niis.org/blog/2018/10/3/x-road-rest-support-where-are-we-today">RFC-ed by NIIS</a> on Oct 3, 2018. For more accurate feedback, I'd also like to see the transport protocol change proposal and get to know some of the reasoning or discussion that led to any particular proposal. For example, I'm not sure whether the receiving proxy<sup id="fnref-proxy"><a href="https://m811.com/articles/comments-on-xroad-protocol-for-rest/#fn-proxy" class="footnote-ref">proxy</a></sup> is also supposed to see the full path as presented below.</p>
<h2 id="message-format">Message Format</h2>
<p>While regular HTTP and REST depend on DNS and hostnames for routing and service locations, X-Road has it's own routing hierarchy: <strong>instance</strong> -> <strong>organization</strong> -> <strong>subsystem</strong>. The proposed method of encoding that information along with the resource (the HTTP-equivalent to SOAP's <strong>service</strong> and <strong>method-call</strong>) is:</p>
<pre><code class="">{http-method} /rest/{version}/{consumer-subsystem}/{provider-subsystem}/{service-id}
</code></pre>
<p>And an example of it with real values:</p>
<pre><code class="">GET /rest/v1/ee/NGO/1234/mine/ee/GOV/5678/theirs/v1/widgets/42
</code></pre>
<p>I'm not convinced it's optimal to encode both the consumer's and the provider's routing information in the URI path. Based on how the proxy server today works, I'm assuming the consumer's information is necessary for it to choose which certificate to sign with and to present to the provider's proxy or service. However, for the provider it's entirely redundant --- the same identity information, with additional possibly valuable fields, is in the message signature. For the consumer you could make the case it's equally redundant majority of the time. For proxy setups for a single consumer-organization, the organization's identity is implicit. For setups that use alternative components, bypassing the client-side proxy, it's also already available and set up (configuring a library to sign with a particular certificate, for example). <em>If</em> the consumer's identity information was implicitly expected to be passed to the provider's business-logic server by the provider's proxy, then I think that's better handled by unified <a href="https://m811.com/articles/comments-on-xroad-protocol-for-rest/#x-headers">x-headers (see below)</a>.</p>
<p>From an aesthetic and REST-ful angle, it also doesn't make much sense to speak about remote services "belonging" to a consumer.</p>
<p>A minor argument, but a result of the hierarchy mismatch, is tying the consumer to the path slightly inconveniences code re-use. Most HTTP libraries permit setting (shared) headers for all requests. They don't usually easily permit modifying the path transparently to the code that constructed the request. In other words, a (shared) function that requests a resource can more easily be re-used if it doesn't have to concern itself with the consumer (sender).</p>
<h3 id="x-headers">X-Headers</h3>
<p>Later sections suggest using X-headers for additional context data, such as the <em>person-id</em> field Estonian X-Road services occasionally expect to see. I think unifying consumer identification with other optional headers is a more elegant solution for two reasons: It permits omitting it for single-organization proxy setups, or for alternative transport components, <em>and</em> permits the receiver's proxy to fill it for the servers it's proxying for. Should they require additional information from the signature (such as time), that could easily be added as additional headers.</p>
<p>While ideally I'd like to see no routing information in the path and see X-Road get out of the ad-hoc name resolving business (use DNS over distributing the <code class="">/etc/hosts</code> equivalent in a megabyte of XML), I'd first like to hear the reasoning behind proposing the path route.</p>
<p>I realize the majority of X-Road users are using the proxy setup today, but I think it's beneficial to think about the protocol separately and then see how to reconcile that with optional proxying.</p>
<h2 id="errors">Errors</h2>
<p>The RFC described limiting errors the proxy may respond with it. It is unclear to me, how are users of the proxy supposed to differentiate between proxy errors and remote server (not remote proxy) errors?</p>
<h2 id="examples">Examples</h2>
<p>I know these were just examples, but I found it a little funny that they already differed from the school of REST that I subscribe to, highlighting either how easy it is to make mistakes <strong>or</strong> how there's no immediate consensus on how to implement REST. It happens the <a href="https://github.com/WhiteHouse/api-standards">White House REST Best Practices</a> link the examples supposedly mimic match what I'd expect, so I'm guessing the protocol examples were mistakes.</p>
<p>Specifically:</p>
<ul>
<li><code class="">GET /pet/{petId}</code> --- falls under the White House's example of a bad URL, in that "pet" is not plural.</li>
<li><code class="">PUT /pet</code> --- Shouldn't update a single pet by the id in the body, but an entire collection (if plural) or a "singleton" pet identified by the URI <code class="">/pet</code>.</li>
<li><code class="">POST /pet/{petId}/uploadImage</code> --- better to have it under <code class="">PUT /pets/42/image</code>. Think [idempotent] actions on resources, not function calls!</li>
</ul>
<div class="footnotes">
<hr />
<ol>
<li id="fn-proxy">I'll use the term "proxy" to refer to "security server" to emphasize there may and are alternative components for performing message signing and transport security. I find that distinction may give rise to more optimal protocol designs.<a href="https://m811.com/articles/comments-on-xroad-protocol-for-rest/#fnref-proxy" class="footnote-backref">↩</a></li>
</ol>
</div>