Government by algorithm

Government by algorithm (also known as Algorithmic regulation, Regulation by algorithms, Algorithmic governance, Algorithmic legal order or Algocracy) is an alternative form of government or social ordering, where the usage of computer algorithms, especially of artificial intelligence and blockchain, is applied to regulations, law enforcement, and generally any aspect of everyday life such as transportation or land registration.[1][2][3][4][5][6][7] Alternatively, algorithmic regulation is defined as setting the standard, monitoring and modification of behaviour by means of computational algorithms — automation of judiciary is in its scope.[8]

Government by algorithm raises new challenges that are not captured in the e-Government literature and the practice of public administration.[9] Some sources equate cyberocracy, which is a hypothetical form of government that rules by the effective use of information,[10][11][12] with algorithmic governance, although algorithms are not the only means of processing information.[13][14] Nello Cristianini and Teresa Scantamburlo argued that the combination of a human society and an algorithmic regulation forms a social machine.[15]

History

In 1962, head of the Department of technical physics in Kiev, Alexander Kharkevich, published an article in the journal "Communist" about a computer network for processing of information and control of economy.[16][17] In fact, he proposed to make a network like the modern Internet for the needs of algorithmic governance.

In 1971–1973, the Chilean government carried out the Project Cybersyn during the presidency of Salvador Allende. This project was aimed at constructing a distributed decision support system to improve the management of the national economy.[18][2]

Also in the 1960s and 1970s, Herbert A. Simon championed expert systems as tools for rationalization and evaluation of administrative behavior.[19] The automation of rule-based processes was an ambition of tax agencies over many decades resulting in varying success.[20] Early work from this period includes Thorne McCarty's influential TAXMAN project[21] in the US and Ronald Stamper's LEGOL project[22] in the UK. The Honourable Justice Michael Kirby published a paper in 1998, where he expressed optimism that the then-available computer technologies such as legal expert system could evolve to computer systems, which will strongly affect the practice of courts.[23] In 2006, attorney Lawrence Lessig known for the slogan "Code is law" wrote:

"[T]he invisible hand of cyberspace is building an architecture that is quite the opposite of its architecture at its birth. This invisible hand, pushed by government and by commerce, is constructing an architecture that will perfect control and make highly efficient regulation possible" [24]

Since 2000s, algorithms are designed and used to automatically analyze surveillance videos.[25]

Overview and Examples

Written laws are not replaced but stressed to test its efficiency. Algorithmic regulation is supposed to be a system of governance where more exact data collected from citizens via their smart devices and computers are used for more efficiency in organizing human life as a collective.[26][27] As Deloitte estimated in 2017, automation of US government work could save 96.7 million federal hours annually, with a potential savings of $3.3 billion; at the high end, this rises to 1.2 billion hours and potential annual savings of $41.1 billion.[28] According to a study of Stanford University, 45% of the studied US federal agencies have experimented with AI and related machine learning (ML) tools up to 2020.[4]

In 2013, algorithmic regulation was coined by Tim O'Reilly, Founder and CEO of O'Reilly Media Inc.:

Sometimes the "rules" aren't really even rules. Gordon Bruce, the former CIO of the city of Honolulu, explained to me that when he entered government from the private sector and tried to make changes, he was told, "That's against the law." His reply was "OK. Show me the law." "Well, it isn't really a law. It's a regulation." "OK. Show me the regulation." "Well, it isn't really a regulation. It's a policy that was put in place by Mr. Somebody twenty years ago." "Great. We can change that!""

[...] Laws should specify goals, rights, outcomes, authorities, and limits. If specified broadly, those laws can stand the test of time. Regulations, which specify how to execute those laws in much more detail, should be regarded in much the same way that programmers regard their code and algorithms, that is, as a constantly updated toolset to achieve the outcomes specified in the laws. [...] It's time for government to enter the age of big data. Algorithmic regulation is an idea whose time has come.[29]

A 2019 poll made by Center for the Governance of Change at IE University in Spain showed that 25% of citizens from selected European countries are somewhat or totally in favor of letting an artificial intelligence make important decisions about the running of their country.[30] Following table shows detailed results:

CountryPercentage
France25%
Germany31%
Ireland29%
Italy28%
Netherlands43%
Portugal19%
Spain26%
UK31%

Use of AI in government agencies

US federal agencies counted the following numbers of artificial intelligence applications.[4]

Agency NameNumber of Use Cases
Office of Justice Programs12
Securities and Exchange Commission10
National Aeronautics and Space Administration9
Food and Drug Administration8
United States Geological Survey8
United States Postal Service8
Social Security Administration7
United States Patent and Trademark Office6
Bureau of Labor Statistics5
U.S. Customs and Border Protection4

53% of these applications were produced by in-house experts.[4] Commercial providers of residual applications include Palantir Technologies.[31] From 2012, NOPD started a secretive collaboration with Palantir Technologies in the field of predictive policing.[32] According to the words of James Carville, he was impetus of this project and "[n]o one in New Orleans even knows about this".[32]

AI politicians

In 2018, an activist named Michihito Matsuda ran for mayor in the Tama city area of Tokyo as a human proxy for an artificial intelligence program.[33] While election posters and campaign material used the term 'robot', and displayed stock images of a feminine android, the 'AI mayor' was in fact a machine learning algorithm trained using Tama city datasets.[34] The project was backed by high-profile executives Tetsuzo Matsumoto of Softbank and Norio Murakami of Google.[35] Michihito Matsuda came third in the election, being defeated by Hiroyuki Abe.[36] Organisers claimed that the 'AI mayor' was programmed to analyze citizen petitions put forward to the city council in a more 'fair and balanced' way than human politicians.[37]

In 2019, AI-powered messenger chatbot SAM participated in the discussions on social media connected to electoral race in New Zealand.[38] The creator of SAM, Nick Gerritsen, believes SAM will be advanced enough to run as a candidate by late 2020, when New Zealand has its next general election.[39]

AI judges

According to the statement of Beijing Internet Court, China is the first country to create an internet court or cyber court.[40][41][42] Chinese AI judge is a virtual recreation of an actual female judge. She "will help the court's judges complete repetitive basic work, including litigation reception, thus enabling professional practitioners to focus better on their trial work".[40]

Also Estonia plans to employ artificial intelligence to decide small-claim cases of less than €7,000.[43]

COMPAS software is used in USA to assess the risk of recidivism in courts.[44][45]

Reputation systems

Tim O'Reilly suggested that data sources and reputation systems combined in algorithmic regulation can outperform traditional regulations.[29] For instance, once taxi-drivers are rated by passengers, the quality of their services will improve automatically and "drivers who provide poor service are eliminated".[29] O'Reilly's suggestion is based on control-theoreric concept of feed-back loopimprovements and disimprovements of reputation enforce desired behavior.[15] The usage of feed-loops for the management of social systems is already been suggested in management cybernetics by Stafford Beer before.[46]

The Chinese Social Credit System is closely related to China's mass surveillance systems such as the Skynet,[47][48][49] which incorporates facial recognition system, big data analysis technology and AI.[50][51][52][53] This system provides assessments of trustworthiness of individuals and businesses.[54][55][56] Among behavior, which is considered as misconduct by the system, jaywalking and failing to correctly sort personal waste are cited.[57][58][59][60][61] Behavior listed as positive factors of credit ratings includes donating blood, donating to charity, volunteering for community services, and so on.[62][63] Chinese Social Credit System enables punishments of "untrustworthy" citizens like denying purchase of tickets and rewards for "trustworthy" citizen like less waiting time in hospitals and government agencies.[64][65][66]

Management of infection

In February 2020, China launched a mobile app to deal with Coronavirus outbreak.[67] Users are asked to enter their name and ID number. The app is able to detect 'close contact' using surveillance data and therefore a potential risk of infection. Every user can also check the status of three other users. If a potential risk is detected, the app not only recommends self-quarantine, it also alerts local health officials.[68]

Cellphone data is used to locate infected patients in South Korea, Taiwan, Singapore and other countries.[69][70] In March 2020, the Israeli government enabled security agencies to track mobile phone data of people supposed to have coronavirus. The measure was taken to enforce quarantine and protect those who may come into contact with infected citizens.[71] Also in March 2020, Deutsche Telekom shared private cellphone data with the federal government agency, Robert Koch Institute, in order to research and prevent the spread of the virus.[72] Russia deployed facial recognition technology to detect quarantine breakers.[73] Italian regional health commissioner Giulio Gallera said that "40% of people are continuing to move around anyway", as he has been informed by mobile phone operators.[74] In USA, Europe and UK, Palantir Technologies is taken in charge to provide COVID-19 tracking services.[75]

Blockchain

Cryptocurrencies, Smart Contracts and Decentralized Autonomous Organization are mentioned as means to replace traditional ways of governance.[76][77][7] Cryptocurrencies are currencies, which are enabled by algorithms without a governmental central bank.[78] Smart contracts are self-executable contracts, whose objectives are the reduction of need in trusted governmental intermediators, arbitrations and enforcement costs.[79][80] A decentralized autonomous organization is an organization represented by smart contracts that is transparent, controlled by shareholders and not influenced by a central government.[81][82][83]

Criticism

The are potential risks associated with the use of algorithms in government. Those include algorithms becoming susceptible to bias,[84] a lack of transparency in how an algorithm may make decisions,[85] and the accountability for any such decisions.[85] There is also a serious concern that gaming by the regulated parties might occur, once more transparency is brought into the decision making by algorithmic governance, regulated parties might try to manipulate their outcome in own favor and even use adversarial machine learning.[4][15] According to Harari, the conflict between democracy and dictatorship is seen as a conflict of two different data-processing systems — AI and algorithms may swing the advantage toward the latter by processing enormous amounts of information centrally.[86] Also, the contributors in the 2019's documentary iHuman express apprehension of "infinitely stable dictatorships" being created by governmental use of AI.[87]

Regulation of algorithmic governance

The Netherlands employed an algorithmic system SyRI (Systeem Risico Indicatie) to detect citizens perceived being high risk for committing welfare fraud, which quietly flagged thousands of people to investigators.[88] This caused a public protest. The district court of Hague shut down SyRI referencing Article 8 of the European Convention on Human Rights (ECHR).[89]

In the USA, multiple states implement predictive analytics as part of their child protection system. Illinois and Los Angeles shut these algorithms down due to a high rate of false positives.[90]

The novels Daemon and Freedom™ by Daniel Suarez describe a fictional scenario of global algorithmic regulation.[91]

See also

  • Government by Algorithm by Stanford University
  • A governance framework for algorithmic accountability and transparency by European parlament
  • Algorithmic Government by Prof. Philip C. Treleaven of University College London
  • Artificial Intelligence for Citizen Services and Government by Hila Mehr of Harvard University
  • Yeung, Karen; Lodge, Martin (2019). Algorithmic Regulation. Oxford University Press. ISBN 9780198838494.
  • Code: Version 2.0 (Basic Books, 2006) ISBN 978-0-465-03914-2
  • iHuman (Documentary, 2019) by Tonje Hessen Schei

References

  1. Yeung, Karen (December 2018). "Algorithmic regulation: A critical interrogation". Regulation & Governance. 12 (4): 505–523. doi:10.1111/rego.12158.
  2. Medina, Eden (2015). "Rethinking algorithmic regulation" (PDF). Kybernetes. 44.6/7 (6/7): 1005–1019. doi:10.1108/K-02-2015-0052.
  3. Katzenbach, Christian; Ulbricht, Lena (29 November 2019). "Algorithmic governance". Internet Policy Review. 8 (4). doi:10.14763/2019.4.1424. ISSN 2197-6775. Retrieved 19 March 2020.
  4. School, Stanford Law. "Government by Algorithm: A Review and an Agenda". Stanford Law School. Retrieved 20 March 2020.
  5. Abril, Rubén Rodríguez. "DERECOM. Derecho de la Comunicación. - An approach to the algorithmic legal order and to its civil, trade and financial projection". www.derecom.com (in Spanish). Retrieved 20 May 2020.
  6. "Rule by Algorithm? Big Data and the Threat of Algocracy". ieet.org. Retrieved 20 May 2020.
  7. Werbach, Kevin (24 September 2018). "The Siren Song: Algorithmic Governance By Blockchain". Social Science Research Network. SSRN 3578610. Cite journal requires |journal= (help)
  8. Hildebrandt, Mireille (6 August 2018). "Algorithmic regulation and the rule of law". Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences. 376 (2128): 20170355. Bibcode:2018RSPTA.37670355H. doi:10.1098/rsta.2017.0355. PMID 30082301.
  9. Veale, Michael; Brass, Irina (2019). "Administration by Algorithm? Public Management Meets Public Sector Machine Learning". Social Science Research Network. SSRN 3375391.
  10. David Ronfeldt (1991). "Cyberocracy, Cyberspace, and Cyberology:Political Effects of the Information Revolution" (PDF). RAND Corporation. Retrieved 12 Dec 2014.
  11. David Ronfeldt (1992). "Cyberocracy is Coming" (PDF). RAND Corporation. Retrieved 12 Dec 2014.
  12. Ronfeldt, David; Varda, Danielle (1 December 2008). "The Prospects for Cyberocracy (Revisited)". Social Science Research Network. SSRN 1325809. Cite journal requires |journal= (help)
  13. "Opinion | Transparency in governance, through cyberocracy". kathmandupost.com. Retrieved 25 April 2020.
  14. Hudson, Alex (28 August 2019). "'Far more than surveillance' is happening and could change how government is run". Metro. Retrieved 25 April 2020.
  15. Cristianini, Nello; Scantamburlo, Teresa (8 October 2019). "On social machines for algorithmic regulation". AI & Society. arXiv:1904.13316. Bibcode:2019arXiv190413316C. doi:10.1007/s00146-019-00917-8. ISSN 1435-5655.
  16. "Machine of communism. Why the USSR did not create the Internet". csef.ru (in Russian). Retrieved 21 March 2020.
  17. Kharkevich, Aleksandr Aleksandrovich (1973). Theory of information. The identification of the images. Selected works in three volumes. Volume 3. Information and technology: Moscow: Publishing House "Nauka", 1973. - Academy of Sciences of the USSR. Institute of information transmission problems. pp. 495–508.
  18. "IU professor analyzes Chile's 'Project Cybersyn'". UI News Room. Archived from the original on 10 September 2009. Retrieved 27 May 2013.
  19. Freeman Engstrom, David; Ho, Daniel E.; Sharkey, Catherine M.; Cuéllar, Mariano-Florentino (2020). "Government by Algorithm: Artificial Intelligence in Federal Administrative Agencies" (PDF).
  20. Margretts, Helen (1999). Information technology in government : Britain and America. New York: Routledge. ISBN 9780203208038.
  21. McCarty, L. Thorne. Reflections on" Taxman: An Experiment in Artificial Intelligence and Legal Reasoning. Harvard Law Review (1977): 837-893.
  22. Stamper, Ronald K. The LEGOL 1 prototype system and language. The Computer Journal 20.2 (1977): 102-108.
  23. Kirby, Michael (1998). "The Future of Courts - Do They Have One". Journal of Law and Information Science. p. 141. Retrieved 12 April 2020.
  24. Lawrence, Lessig (2006). Code (Version 2.0 ed.). Basic Books. ISBN 978-0-465-03914-2.
  25. Sodemann, Angela A.; Ross, Matthew P.; Borghetti, Brett J. (November 2012). "A Review of Anomaly Detection in Automated Surveillance". IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews). 42 (6): 1257–1272. doi:10.1109/TSMCC.2012.2215319.
  26. McCormick, Tim. "A brief exchange with Tim O'Reilly about "algorithmic regulation" | Tim McCormick". Retrieved 2 June 2020.
  27. "Why the internet of things could destroy the welfare state". the Guardian. 19 July 2014. Retrieved 2 June 2020.
  28. Eggers, illiam D.; Schatsky, David; Viechnick, Peter. "Demystifying artificial intelligence in government | Deloitte Insights". www2.deloitte.com. Retrieved 4 April 2020.
  29. O’Reilly, Tim (2013). "Open Data and Algorithmic Regulation". In Goldstein, B.; Dyson, L. (eds.). Beyond Transparency: open Data and the Future of Civic Innovation. San Francisco: Code for America Press. pp. 289–300.
  30. "European Tech Insights (2019) | IE CGC" (PDF). Center for the Governance of Change. Retrieved 11 April 2020.
  31. "Leaked Palantir Doc Reveals Uses, Specific Functions And Key Clients". TechCrunch. Retrieved 22 April 2020.
  32. Winston, Ali (27 February 2018). "Palantir has secretly been using New Orleans to test its predictive policing technology". The Verge. Retrieved 23 April 2020.
  33. "POLITICS 2028: WHY ARTIFICIAL INTELLIGENCE WILL REPLACE POLITICIANS b…". SlideShare. 14 July 2018. Retrieved 22 September 2019.
  34. Johnston, Lachlan (12 April 2018). "There's an AI Running for the Mayoral Role of Tama City, Tokyo". OTAQUEST. Retrieved 22 September 2019.
  35. "AI党 | 多摩市議会議員選挙2019". AI党 | 多摩市議会議員選挙2019.
  36. Robot's mayoral race: AI candidate gets thousands of votes in Japanese city
  37. O'Leary, Abigail; Verdon, Anna (April 17, 2018). "Robot to run for mayor in Japan promising 'fairness and balance' for all". mirror.
  38. Sarmah, Harshajit (28 January 2019). "World's First AI-powered Virtual Politician SAM Joins The Electoral Race In New Zealand". Analytics India Magazine. Retrieved 11 April 2020.
  39. "Meet SAM, world's first AI politician that hopes to run for New Zealand election in 2020". Hindustan Times. 26 November 2017. Retrieved 11 April 2020.
  40. "Beijing Internet Court launches online litigation service center". english.bjinternetcourt.gov.cn. Retrieved 13 April 2020.
  41. "China Now Has AI-Powered Judges". RADII | Culture, Innovation, and Life in today's China. 16 August 2019. Retrieved 13 April 2020.
  42. Fish, Tom (6 December 2019). "AI shock: China unveils 'cyber court' complete with AI judges and verdicts via chat app". Express.co.uk. Retrieved 13 April 2020.
  43. "Can AI Be a Fair Judge in Court? Estonia Thinks So". Wired. Retrieved 13 April 2020.
  44. Sam Corbett-Davies, Emma Pierson, Avi Feller and Sharad Goel (October 17, 2016). "A computer program used for bail and sentencing decisions was labeled biased against blacks. It's actually not that clear". The Washington Post. Retrieved January 1, 2018.CS1 maint: multiple names: authors list (link)
  45. Aaron M. Bornstein (December 21, 2017). "Are Algorithms Building the New Infrastructure of Racism?". Nautilus. No. 55. Retrieved January 2, 2018.
  46. Beer, Stafford (1975). Platform for change : a message from Stafford Beer. J. Wiley. ISBN 978-0471948407.
  47. "最高法打造"天网"破解执行难 去年615万老赖被"限行"".
  48. ""Skynet" China's massive video surveillance network". Abacus. October 4, 2018. Retrieved 2019-11-13.
  49. "中国天网已建成 2亿摄像头毫秒级寻人". Phoenix New Media Limited (in Chinese). May 4, 2018.
  50. "How China's Surveillance State Reflects 'Black Mirror'". Fortune. March 27, 2019. Retrieved 2020-01-11.
  51. "How China Is Using Big Data to Create a Social Credit Score". Time. 2019. Retrieved 2020-01-11.
  52. Lakshmanan, Ravie (2019-09-30). "China's new 500-megapixel 'super camera' can instantly recognize you in a crowd". The Next Web. Retrieved 2019-11-13.
  53. "Chinese city with world's heaviest surveillance has 2.5 million cameras". South China Morning Post. 2019-08-19. Retrieved 2019-11-13.
  54. Nicole Kobie (7 June 2019). "The complicated truth about China's social credit system". WIRED UK.
  55. Stevenson, Alexandra; Mozur, Paul (2019-09-22). "China Scores Businesses, and Low Grades Could Be a Trade-War Weapon". The New York Times. ISSN 0362-4331. Retrieved 2020-01-11.
  56. Cheng, Evelyn (2019-09-04). "China is building a 'comprehensive system' for tracking companies' activities, report says". CNBC. Retrieved 2020-01-11.
  57. 不开玩笑!11月1日起,行人闯红灯和这些违法行为将纳入征信体系 (in Chinese). Shenzhen News. Retrieved 2019-11-09.
  58. 7月8日起 在南京一年闯红灯5次以上将记入个人信用记录 (in Chinese). Sina. 2019-07-07. Retrieved 2019-11-09.
  59. Huang, Echo. "Garbage-sorting violators in China now risk being punished with a junk credit rating". Quartz. Retrieved 2019-11-10.
  60. 《上海市生活垃圾管理条例》全文公布 7月1日起施行 (in Chinese). Sina. Retrieved 2019-11-09.
  61. 新《杭州市生活垃圾管理条例》审批通过垃圾不分类乱丢乱扔将被罚款并计入信用档案_省内要闻_平安浙江网. www.pazjw.gov.cn (in Chinese). Retrieved 2019-11-12.
  62. "How China's 'social credit' system blocked millions of people from travelling". Canadian Broadcasting Corporation. March 7, 2019.
  63. West, Jack Karsten and Darrell M. (2018-06-18). "China's social credit system spreads to more daily transactions". Brookings. Retrieved 2019-11-13.
  64. 2682万人次因失信被限制乘机. XinhuaNet (in Chinese). Retrieved 2019-11-10.
  65. "Social credit system will not stop people using public services, Beijing says". South China Morning Post. 2019-07-19. Retrieved 2019-11-10.
  66. Ma, Alexandra. "China's controversial social credit system isn't just about punishing people — here's what you can do to get rewards, from special discounts to better hotel rooms". Business Insider. Retrieved 2019-11-12.
  67. "China launches coronavirus 'close contact' app". BBC News. 11 February 2020. Retrieved 7 March 2020.
  68. Chen, Angela. "China's coronavirus app could have unintended consequences". MIT Technology Review. Retrieved 7 March 2020.
  69. Manancourt, Vincent (10 March 2020). "Coronavirus tests Europe's resolve on privacy". POLITICO. Retrieved 20 March 2020.
  70. Coronavirus mobile appsare surging in popularity in South Korea
  71. Tidy, Joe (17 March 2020). "Coronavirus: Israel enables emergency spy powers". BBC News. Retrieved 18 March 2020.
  72. Paksoy, Yunus. "German telecom giant shares private data with government amid privacy fears". trtworld. Retrieved 20 March 2020.
  73. "Moscow deploys facial recognition technology for coronavirus quarantine". Reuters. 21 February 2020. Retrieved 20 March 2020.
  74. "Italians scolded for flouting lockdown as death toll nears 3,000". Pittsburgh Post-Gazette. Retrieved 20 March 2020.
  75. "Palantir provides COVID-19 tracking software to CDC and NHS, pitches European health agencies". TechCrunch. Retrieved 22 April 2020.
  76. Bindra, Jaspreet (30 March 2018). "Transforming India through blockchain". Livemint. Retrieved 31 May 2020.
  77. Finn, Ed (10 April 2017). "Do digital currencies spell the end of capitalism?". The Guardian. Retrieved 31 May 2020.
  78. Reiff, Nathan. "Blockchain Explained". Investopedia. Retrieved 31 May 2020.
  79. Szabo, Nick (1997). "View of Formalizing and Securing Relationships on Public Networks | First Monday". firstmonday.org.
  80. Fries, Martin; P. Paal, Boris (2019). Smart Contracts (in German). Mohr Siebeck. ISBN 978-3-16-156911-1. JSTOR j.ctvn96h9r.
  81. "What is DAO - Decentralized Autonomous Organizations". BlockchainHub. Retrieved 31 May 2020.
  82. Prusty, Narayan (27 Apr 2017). Building Blockchain Projects. Birmingham, UK: Packt. p. 9. ISBN 9781787125339.
  83. Chohan, Usman W. (4 December 2017). "The Decentralized Autonomous Organization and Governance Issues". Social Science Research Network. SSRN 3082055. Cite journal requires |journal= (help)
  84. Mehr, Hila (August 2017). "Artificial Intelligence for Citizen Services and Government" (PDF). ash.harvard.edu. Retrieved 2018-12-31.
  85. Capgemini Consulting (2017). "Unleashing the potential of Artificial Intelligence in the Public Sector" (PDF). www.capgemini.com. Retrieved 2018-12-31.
  86. Harari, Story by Yuval Noah. "Why Technology Favors Tyranny". The Atlantic. Retrieved 11 April 2020.
  87. Wilkinson, Amber. "'iHuman': IDFA Review". Screen. Retrieved 21 April 2020.
  88. "Europe Limits Government by Algorithm. The US, Not So Much". Wired. Retrieved 11 April 2020.
  89. Haag, Rechtbank Den (6 March 2020). "ECLI:NL:RBDHA:2020:1878, Rechtbank Den Haag, C-09-550982-HA ZA 18-388 (English)". uitspraken.rechtspraak.nl (in Dutch). Retrieved 11 April 2020.
  90. "NashReport.pdf". Google Docs. Retrieved 13 April 2020.
  91. Rieger, Frank. "Understanding the Daemon". FAZ.NET (in German). Retrieved 5 April 2020.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.