makyo
/
zk_html
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

update from sparkleup

This commit is contained in:
Madison Scott-Clary 2022-03-27 21:15:13 -07:00
parent d991fc31f5
commit 6dba11e372
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
writing/3/unknown-things/mfa/conclusion.html
View File

@ -12,7 +12,7 @@
<h1>Zk | conclusion</h1> <h1>Zk | conclusion</h1>
</header> </header>
<article class="content"> <article class="content">
<h1 id="should-all-things-be-known">Should all things be known</h1> <h1 class="unnumbered" id="should-all-things-be-known">Should all things be known</h1>
<p>There is a concept that I&rsquo;ve seen explored a few times and in a few different ways called &ldquo;instrumental convergence&rdquo;. It&rsquo;s this idea that there is a tendency to pursue infinite goals despite the bounded nature of the available resources. In order to construct the instruments required to achieve what may indeed be infinite, there&rsquo;s the risk that all resources may be consumed in the process.</p> <p>There is a concept that I&rsquo;ve seen explored a few times and in a few different ways called &ldquo;instrumental convergence&rdquo;. It&rsquo;s this idea that there is a tendency to pursue infinite goals despite the bounded nature of the available resources. In order to construct the instruments required to achieve what may indeed be infinite, there&rsquo;s the risk that all resources may be consumed in the process.</p>
<p>I actually learned about this through the delightful example of the clicker game Universal Paperclips, which is a rather on the nose exploration of the paperclip maximizer thought experiment.</p> <p>I actually learned about this through the delightful example of the clicker game Universal Paperclips, which is a rather on the nose exploration of the paperclip maximizer thought experiment.</p>
<p>This thought experiment and its implementation in Universal Paperclips states that, should a very single-minded AI be provided with the sole goal of maximizing the number of paperclips that it can make will first aim to increase the speed at which it can do so. Perhaps then it will include the ability to auto-buy wire so that it never runs out. That can get expensive, though, so perhaps it starts investing heavily in order to fund this, and then heck, maybe it starts fiddling with the markets behind the scenes.</p> <p>This thought experiment and its implementation in Universal Paperclips states that, should a very single-minded AI be provided with the sole goal of maximizing the number of paperclips that it can make will first aim to increase the speed at which it can do so. Perhaps then it will include the ability to auto-buy wire so that it never runs out. That can get expensive, though, so perhaps it starts investing heavily in order to fund this, and then heck, maybe it starts fiddling with the markets behind the scenes.</p>