Jekyll2018-10-02T23:55:07+00:00https://robertmaxwilliams.github.io//Robert Max WilliamsAI and AI safety enthusiastPreprint of Optical Illusions Paper2018-10-01T00:00:00+00:002018-10-01T00:00:00+00:00https://robertmaxwilliams.github.io//Preprint<p>With the help of Dr. Yampolskiy, I put the information in the series on optical illusions together
into an academic paper, which can be found here: <a href="https://arxiv.org/abs/1810.00415">https://arxiv.org/abs/1810.00415</a></p>
<p>The main goal of creating the dataset is more or less complete, but as it stands it is not super
useful and could use some reorganization.</p>
<p>I sent the paper to Luke Miles to see what he thought (after submitting to Arxiv, unfortunately)
and he gave the following feedback:</p>
<p>Unrequested feedback on Max’s paper:</p>
<ul>
<li>That was actually a good read</li>
<li>The 1 footnote on Max’s name is never resolved</li>
<li>First couple sentences of abstract are not relevant</li>
<li>Backwards quotes?</li>
<li>Italic author names?</li>
<li>Is confusion matrix test or train?</li>
<li>Column labels on confusion matrix?</li>
<li>For consistency, section 4.2 should be called “Generative Adversarial Network Results”</li>
<li>In section 5, clarify that the vision system of interest is the eyes of
<em>predators</em>. Even if butterflies couldn’t see, they would evolve fake eyes
to fool bluejays. Feedback is via the killing of butterflies (by owls) who
wear weak illusions.</li>
<li>Good choice to host the images on floydhub; I had like 8MB/S download speed.
Maybe clarify that you can download them for free without an account, because
floydhub may try to get them to sign up first (like on dropbox)</li>
</ul>
<p>I didn’t realize latex required specifying quote direction, so all of my quotes are backwards! How
embarrassing. I won’t re-upload for cosmetic changes because that’s against Arxiv’s archiving
philosophy, but when I do make another version I will incorporate these changes.</p>With the help of Dr. Yampolskiy, I put the information in the series on optical illusions together into an academic paper, which can be found here: https://arxiv.org/abs/1810.00415Data Download and Initial Neural Network Training Results2018-04-10T00:00:00+00:002018-04-10T00:00:00+00:00https://robertmaxwilliams.github.io//Download-Time<p>I downloaded and cleaned all of the files from <a href="https://www.moillusions.com/4-dots-illusion/">Mighty Optical Illusions</a> and <a href="http://viperlib.york.ac.uk/">ViperLib</a> into JPEG (.jpg) images. They are available for download from <a href="https://www.floydhub.com/robertmax/datasets/illusions-jpg">https://www.floydhub.com/robertmax/datasets/illusions-jpg</a> and the source files and build process can be found on <a href="https://github.com/robertmaxwilliams/optical-illusion-dataset">https://github.com/robertmaxwilliams/optical-illusion-dataset</a>.</p>
<p>A greatly reduced version of around 500 images can be found in <a href="https://www.floydhub.com/robertmax/datasets/illusions-filtered">this floydhub dataset</a>. Training a GAN on these might yield better results than the last attempt with the full dataset (see below).</p>
<h3 id="classifier">Classifier</h3>
<p>I trained the “<a href="https://www.tensorflow.org/tutorials/image_retraining">bottleneck</a>” of an image classification model, taken from the tensorflow models repo, on the moillusion images. I used a smaller subset of the categories, categories that had at least 20 images and seemed relevant to the goals of the project.</p>
<p><img src="/images/confusion-illusion.png" alt="better than random" /></p>
<p>It performed much better than random, but not very well. I doubt it learned anything meaningful to the data, only texture and context clues that roughly correlate with the assigned classes. As well, the data is multi-class but I treated it as single class by including an images in all of its given classes. This means that on a multiclass image, it was guessing the class. I didn’t account for this in the results, which I really should have.</p>
<p>I need to repeat this study with a proper multiclass model. I want to make it from scratch in Keras or Tensorflow, but will need more computing power to train the full image convolution layers instead of just the dense bottleneck layers which I trained on my laptop.</p>
<h3 id="gan-failure">GAN Failure</h3>
<p>This is the second time I’ve attempted to train a GAN. The first was on kickstarter images, and led to some strange shapes and textures, but nothing sellable. This time I used the full dataset linked above, and got similar results. It learns… something. It’s hard to say what exactly. One thing I realized in hindsight is that later in training, every image in the group looks exactly the same. A lack of variety within a batch indicates some sort of mistake on my part setting the model parameters, not just with the data.</p>
<p><img src="/images/000001.png" alt="Not good" />
<img src="/images/000263.png" alt="Better" />
<img src="/images/000365.png" alt="Actually, not good at all" />
<strong>training progression of the hyperGAN model on the viperlib images</strong>_</p>
<p>I only attempted one full training run, and let it go overnight until I used up all of my floydhub credits. Dr. Yampolskiy suggested that I could use the university’s computing resources, which would make it practical to experiment more with these computationally expensive models. With more experimentation and a more narrow dataset, I might actually get a model to generate images that create an illusion, instead of the illusion-of-being-colorful-blobs that GAN models are so eager to produce.</p>
<h3 id="whats-next">What’s Next</h3>
<p>I’m working over the summer on a machine learning project (details soon) and should be able to apply my new skills to this project. So far I’ve spent most of my time working with the data, and very little tweaking the actual models. Now that I have a nice clean dataset and two baseline projects, I can improve upon it from here. The data work isn’t totally done; the disparity between viperlib and moillusions is a difficult problem that needs to be resolved thoughfully. A full human labeling of the dataset would be nice, but I would need to recruit help to make high quality labeling feasible, and make software to streamline the process.</p>I downloaded and cleaned all of the files from Mighty Optical Illusions and ViperLib into JPEG (.jpg) images. They are available for download from https://www.floydhub.com/robertmax/datasets/illusions-jpg and the source files and build process can be found on https://github.com/robertmaxwilliams/optical-illusion-dataset.Long Term Universe Outcomes2018-02-14T00:00:00+00:002018-02-14T00:00:00+00:00https://robertmaxwilliams.github.io//Long_Term_Universe_Outcoes<p>The Fermi Paradox is a huge problem for humanity. The simple question, “Where are they?” has no easy answer. Are there really no aliens out there colonizing the universe? Maybe life is unlikely, or maybe no life ever gets off its home planet. Or perhaps alien life is crowding the galaxy but we are too small minded to see it or are intentionally blocked off by a higher order. Assuming the observable universe is as cold and dead as it looks, humans have a few options going forward:</p>
<p><strong>Team Human</strong> : Humans make it to control the universe while remaining recognizable and fill the universe with corn and war and video games.</p>
<p><strong>Everyone Dies</strong> : The Unsiverse is dead again, Earth is still likely to breed another smart monkey in a few billion years, if no one else gets to it. Go back 4 spaces.</p>
<p><strong>Nanotech</strong> : Tiny robots eat everything! They might colonize the universe and even evolve into interesting and complex forms of life 2.0, but are completely alien so we’re not happy about that. Do not pass go, do not collect 200 dollars.</p>
<p><strong>Paperclip maximizer</strong> : AI pursuing a goal eats the universe and maximizes its reward. The fallout might have interesting dynamics that harbor life 2.0 of some sort, but the long term outcome is that the universe is converted to things of little human value.</p>
<p><strong>Human maximizer</strong> : An AI fills the universe with <a href="https://www.goodreads.com/quotes/1413237-consider-an-ai-that-has-hedonism-as-its-final-goal">hedonium</a>, not much better than paperclips.</p>
<p><strong>VR</strong> : We live in virtual reality. Humanity either stay confined to the solar system or fill the universe, effectively hedonium.</p>
<p><strong>Humans 2.0</strong> : We augment ourselves with machines, maybe even become a hivemind. We remain human and fill the universe with beauty and science while remaining in control.</p>
<p><strong>Invent God</strong> : Friendly AI helps us make the universe into our happy but challenging garden and everyone lives happily ever after as their ideal self, but the universe is controlled by non-human entity.</p>
<p>This list is incomplete, and I am no authority in determining humanities long term future. However, I do think the stakes are very high, and the long term outcome for the observable universe depends on our actions as a species. Notice that only a few of these outcomes result in humans leaving Earth, and it is up to us to prove ourselves worthy of the prize that is the universe for all of time. Tech arms races point us towards the less attrctive items on this list, so let’s try to keep things under control with this “AI arms race” that we seems to be on the verge of.</p>
<p>Thanks to <a href="http://lukemiles.org/">Luke Miles</a> for the discussion and idea to make this list into a blog post.</p>The Fermi Paradox is a huge problem for humanity. The simple question, “Where are they?” has no easy answer. Are there really no aliens out there colonizing the universe? Maybe life is unlikely, or maybe no life ever gets off its home planet. Or perhaps alien life is crowding the galaxy but we are too small minded to see it or are intentionally blocked off by a higher order. Assuming the observable universe is as cold and dead as it looks, humans have a few options going forward:Fun With (Fun With Quines in Python) Quines In Python2018-02-09T00:00:00+00:002018-02-09T00:00:00+00:00https://robertmaxwilliams.github.io//Quine-Fun<p>I’ve seen quines before, as well as quines that aren’t really quines, like this dumb bash trick:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">echo</span> <span class="nv">$BASH_COMMAND</span>
</code></pre></div></div>
<p>However, I couldn’t really get my head around how to write one. I decided to give it a shot using python and see what I could do. I struggled for a bit and went down several tunnels of infinite regression of quotes and escape characters.</p>
<p>I gave in and searched for “python quines” and peeked at the first page for a hint. I noticed that all of the quines used an interediate variable, so I went back to work with that one piece of information. A few more attempts at inifinitely nested quotations later, I came up with this monster:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">q</span> <span class="o">=</span> <span class="s">"</span><span class="se">\"</span><span class="s">"</span>
<span class="n">n</span> <span class="o">=</span> <span class="s">"</span><span class="se">\n</span><span class="s">"</span>
<span class="n">s</span> <span class="o">=</span> <span class="s">"</span><span class="se">\\</span><span class="s">"</span>
<span class="n">m1</span> <span class="o">=</span> <span class="s">'q = '</span><span class="o">+</span><span class="n">q</span><span class="o">+</span><span class="n">s</span><span class="o">+</span><span class="n">q</span><span class="o">+</span><span class="n">q</span><span class="o">+</span><span class="n">n</span><span class="o">+</span><span class="s">'n = '</span><span class="o">+</span><span class="n">q</span><span class="o">+</span><span class="n">s</span><span class="o">+</span><span class="s">'n'</span><span class="o">+</span><span class="n">q</span><span class="o">+</span><span class="n">n</span><span class="o">+</span><span class="s">'s = '</span><span class="o">+</span><span class="n">q</span><span class="o">+</span><span class="n">s</span><span class="o">+</span><span class="n">s</span><span class="o">+</span><span class="n">q</span>
<span class="n">m2</span> <span class="o">=</span> <span class="s">"'q = '+q+s+q+q+n+'n = '+q+s+'n'+q+n+'s = '+q+s+s+q"</span>
<span class="n">foo</span> <span class="o">=</span> <span class="s">"foo = ... print(m1 + n + n + 'm1 = ' + m2 + n + 'm2 = ' + q + m2 + q + n + foo[:6] + q + foo + q + n + foo[10:])"</span>
<span class="k">print</span><span class="p">(</span><span class="n">m1</span> <span class="o">+</span> <span class="n">n</span> <span class="o">+</span> <span class="n">n</span> <span class="o">+</span> <span class="s">'m1 = '</span> <span class="o">+</span> <span class="n">m2</span> <span class="o">+</span> <span class="n">n</span> <span class="o">+</span> <span class="s">'m2 = '</span> <span class="o">+</span> <span class="n">q</span> <span class="o">+</span> <span class="n">m2</span> <span class="o">+</span> <span class="n">q</span> <span class="o">+</span> <span class="n">n</span> <span class="o">+</span> <span class="n">foo</span><span class="p">[:</span><span class="mi">6</span><span class="p">]</span> <span class="o">+</span> <span class="n">q</span> <span class="o">+</span> <span class="n">foo</span> <span class="o">+</span> <span class="n">q</span> <span class="o">+</span> <span class="n">n</span> <span class="o">+</span> <span class="n">foo</span><span class="p">[</span><span class="mi">10</span><span class="p">:])</span>
</code></pre></div></div>
<p><em>Credit to <a href="https://github.com/qpwo">Luke Miles</a> for help spotting that 10 which was an 11 and for motivation.</em></p>
<p>Which I quickly brought down to two lines when I realized what chr() does in python:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">foo</span> <span class="o">=</span> <span class="s">"foo = ... print(foo[:6] + chr(34) + foo + chr(34) + chr(10) + foo[10:])"</span>
<span class="k">print</span><span class="p">(</span><span class="n">foo</span><span class="p">[:</span><span class="mi">6</span><span class="p">]</span> <span class="o">+</span> <span class="nb">chr</span><span class="p">(</span><span class="mi">34</span><span class="p">)</span> <span class="o">+</span> <span class="n">foo</span> <span class="o">+</span> <span class="nb">chr</span><span class="p">(</span><span class="mi">34</span><span class="p">)</span> <span class="o">+</span> <span class="nb">chr</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span> <span class="o">+</span> <span class="n">foo</span><span class="p">[</span><span class="mi">10</span><span class="p">:])</span>
</code></pre></div></div>
<p>And the need to run it easily from plaintext motivated a pseudo-oneliner that only uses single quotes:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">foo</span> <span class="o">=</span> <span class="s">'foo = ... print(foo[:6] + chr(39) + foo + chr(39) + chr(59) + foo[10:])'</span><span class="p">;</span><span class="k">print</span><span class="p">(</span><span class="n">foo</span><span class="p">[:</span><span class="mi">6</span><span class="p">]</span> <span class="o">+</span> <span class="nb">chr</span><span class="p">(</span><span class="mi">39</span><span class="p">)</span> <span class="o">+</span> <span class="n">foo</span> <span class="o">+</span> <span class="nb">chr</span><span class="p">(</span><span class="mi">39</span><span class="p">)</span> <span class="o">+</span> <span class="nb">chr</span><span class="p">(</span><span class="mi">59</span><span class="p">)</span> <span class="o">+</span> <span class="n">foo</span><span class="p">[</span><span class="mi">10</span><span class="p">:])</span>
</code></pre></div></div>
<p>Which leads finally to the single line that can be run straight from the command line as my times as you want:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">python</span> <span class="o">-</span><span class="n">c</span> <span class="s">"foo = 'python -c foo = ... print(foo[:10] + chr(34) + foo[10:16] + chr(39) + foo + chr(39) + chr(59) + foo[20:] + chr(34))';print(foo[:10] + chr(34) + foo[10:16] + chr(39) + foo + chr(39) + chr(59) + foo[20:] + chr(34))"</span>
</code></pre></div></div>
<p>So what is going on and why are these possible? Are they possible in every programming language that is turing complete and has text output? The answer is yes, and I have not idea why. According the wikipedia it has something to do with a Mr. Kleene and his Recursion Theorom, where a quine would be considered a fixed point of the function that is the interpreter. I’ll get back with you when I understand how it can be proved that fixed points always exist.</p>
<p>Until then, here is a quine that also includes my favorite python error message:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">from</span> <span class="nn">sys</span> <span class="kn">import</span> <span class="n">setrecursionlimit</span><span class="p">,</span> <span class="n">stdout</span>
<span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="n">setrecursionlimit</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
<span class="n">foo</span><span class="p">(</span><span class="n">x</span><span class="o">+</span><span class="mi">1000</span><span class="p">)</span>
<span class="n">bar</span> <span class="o">=</span> <span class="s">"""from sys import setrecursionlimit, stdout
def foo(x):
setrecursionlimit(x)
foo(x+1000)
bar = ...
print(bar[:97]+chr(34)*3+bar+chr(34)*3+chr(10)*2+bar[101:]+chr(10)+chr(35), end=str())
stdout.flush()
foo(42)"""</span>
<span class="k">print</span><span class="p">(</span><span class="n">bar</span><span class="p">[:</span><span class="mi">97</span><span class="p">]</span><span class="o">+</span><span class="nb">chr</span><span class="p">(</span><span class="mi">34</span><span class="p">)</span><span class="o">*</span><span class="mi">3</span><span class="o">+</span><span class="n">bar</span><span class="o">+</span><span class="nb">chr</span><span class="p">(</span><span class="mi">34</span><span class="p">)</span><span class="o">*</span><span class="mi">3</span><span class="o">+</span><span class="nb">chr</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span><span class="o">*</span><span class="mi">2</span><span class="o">+</span><span class="n">bar</span><span class="p">[</span><span class="mi">101</span><span class="p">:]</span><span class="o">+</span><span class="nb">chr</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span><span class="o">+</span><span class="nb">chr</span><span class="p">(</span><span class="mi">35</span><span class="p">),</span> <span class="n">end</span><span class="o">=</span><span class="nb">str</span><span class="p">())</span>
<span class="n">stdout</span><span class="o">.</span><span class="n">flush</span><span class="p">()</span>
<span class="n">foo</span><span class="p">(</span><span class="mi">42</span><span class="p">)</span>
<span class="c">#Segmentation fault (core dumped)</span>
</code></pre></div></div>
<p>The program appears on screen easily, but reading the output into a file is more difficult. Using bash redirects and some other <a href="https://stackoverflow.com/questions/23954607/redirection-of-a-out-is-not-capturing-segmentation-fault">wizard</a> <a href="https://askubuntu.com/questions/420981/how-do-i-save-terminal-output-to-a-file">stuff</a> I made this command to save the output to a file:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">{</span> python3 quinefault.py <span class="o">></span> quinefault2.py<span class="p">;</span> <span class="o">}</span> 2>> quinefault2.py
</code></pre></div></div>
<h2 id="i-dont-understand-fixed-points">I Don’t Understand Fixed Points</h2>
<p>Do hash algorithms have fixed points? Encryption? What about f(x) = x+1, it seems like many computable functions don’t have an x where f(x) = x. Maybe I am misunderstanding what is meant by computable function.</p>
<h2 id="taking-the-whole-thing-too-far">Taking the Whole Thing Too Far</h2>
<p>Using a compiler/interpreter/etc as a function and finding a fixed point in it is neat and all, but what about neural networks? Can a neural net output its own weights? Most neural nets used today are capable of approximating universal computation, so shouldn’t they be able to take in a zero vector and output the values of their weights? Using a plain feedforward network would be impossible because the output layer would make the net too large and the whole thing can’t fit inside itself. A recurrent network or some sort of external attention mechanism would be needed. I think such a system would count as a quine. Even though it is supported by a great deal of external equipment, that shouldn’t disqualify it. A python quine doesn’t need to output python’s source, and a neural quine doesn’t need to output its own textual source code. Not saying that it couldn’t…</p>I’ve seen quines before, as well as quines that aren’t really quines, like this dumb bash trick: echo $BASH_COMMANDData Collection Results2018-02-04T00:00:00+00:002018-02-04T00:00:00+00:00https://robertmaxwilliams.github.io//Data-Collection<p>None of the website owners replied to my emails, so I collected the images myself. All of the content on these sites has been collected from other sources, so I don’t think there is any issue with copyright. I have collected all of the image URLs and some metadata for every illusion images on both websites, and some percent of non-illusion images.</p>
<h2 id="mighty-optical-illusions"><a href="https://www.moillusions.com/4-dots-illusion/">Mighty Optical Illusions</a></h2>
<p>To obtain images, I started at this page <a href="https://www.moillusions.com/one-two-face-illusion/">https://www.moillusions.com/one-two-face-illusion/</a>. I used python’s Beuatiful Soup library to copy all image links and then follow the previous button to the next page, and repeated until I was out of next pages. This leads to a grand total of <strong>6436 images</strong>. It’s hard to know how many are duds, but that is a good start.</p>
<h2 id="viperlib"><a href="http://viperlib.york.ac.uk/">ViperLib</a></h2>
<p>Here I changed the page number in the url, and all of the relevant areas on the site. I was able to get <strong>1454</strong> images. The descriptions might end up being useful weeding out some of the illusions that require video.</p>
<h2 id="reddit">Reddit</h2>
<p>On closer inspection, there are only a hundred or so good illusions here. I do not plan to download them.</p>
<p>I think between these two sites I have about as many illusion images as I’ll get. I am waiting to hear back from <a href="http://illusionoftheyear.com/">http://illusionoftheyear.com/</a> to see if they would share their images for the good of science, but I have not heard back from them. The JSON files and scraping code will be available here: <a href="https://github.com/robertmaxwilliams/optical-illusion-dataset">https://github.com/robertmaxwilliams/optical-illusion-dataset</a>. The next step is to combine the JSON files and actually download all of the images and find a place to host them.</p>
<h2 id="next-steps">Next Steps</h2>
<p>I was able to obtain <strong>7890</strong> images from my top two websites, and maybe a few more thousand if IllusionsOfTheYear wants to help.
I plan to learn to use a Variational Autoencoder and apply it to this data. It should be able to come up with a good latent space for these images, and then I can cluster them and see what sorts of clusters appear. At this point I can try training a GAN on specific classes, perhaps certain illusions are easy for a GAN to reproduce than others. I expect that texture based geometric illusions will be obtainable, but anything with large scale structure will not be possible without a major revolution in AI or a few orders of magnitude more images.w</p>None of the website owners replied to my emails, so I collected the images myself. All of the content on these sites has been collected from other sources, so I don’t think there is any issue with copyright. I have collected all of the image URLs and some metadata for every illusion images on both websites, and some percent of non-illusion images.Optical Illusion Datasets2018-01-15T00:00:00+00:002018-01-15T00:00:00+00:00https://robertmaxwilliams.github.io//Illusion_Dataset<p>I have been digging around for optical illusion datasets. They don’t seem to exist, so I will be laying the grounds for the creation of one and starting to collect data. I don’t know all that much about copyright, so if anything I describe sounds legally dubious I would really like to know.</p>
<p>Things that need to be done:</p>
<ul>
<li>Find enough data to make a good dataset</li>
<li>Clean it up, convert everything to the same format (png or jpeg?) and naming convention and organize metadata somehow</li>
<li>Create different resolution versions and cropped version for easier usage</li>
<li>Make it into an easy to use archive and host in on UofL server or github or something</li>
</ul>
<h2 id="find-enough-data-for-a-good-dataset">Find Enough Data for a Good Dataset</h2>
<h3 id="httpwwwmichaelbachdeot"><a href="http://www.michaelbach.de/ot/">http://www.michaelbach.de/ot/</a></h3>
<p>★★★★☆</p>
<p>This website is really amazing, but many of his illusions require video to work. Current ML research on these moving pictures is limited, including videos would likely be a waste of time for the current state of the art. If we exclude videos, this website only has a few dozen images, possible worth collecting if we need it.</p>
<h3 id="httpswwwmoillusionscom"><a href="https://www.moillusions.com/">https://www.moillusions.com/</a></h3>
<p>★★★★★</p>
<p>The bottom bar shows 414 pages, and each page has 8 posts, which appear to be a short article and a high quality illusion image each. Not only does this mean we have 3312 images, we also have textual metadata and categories! This is really something. Probably not enough to train a GAN but maybe some revolution will bring <a href="https://arxiv.org/pdf/1710.09829.pdf">capsule nets</a> to general image generation. Besides, not having enough data is no reason not to try anyways and see what sorts of horrible blobs it generates.
If this is the end-all beat-all source of data, it should be enough to train a classifier and then we can inspect it to see what it believes makes illusions what they are and perhaps find some new insights into human vision.</p>
<h3 id="httpwww-bcsmitedugazpublicationsgazzandirgazzanhtm"><a href="http://www-bcs.mit.edu/gaz/publications/gazzan.dir/gazzan.htm">http://www-bcs.mit.edu/gaz/publications/gazzan.dir/gazzan.htm</a></h3>
<p>★★★☆☆</p>
<p>This article explains various illusions where the same color looks different in different contexts</p>
<h3 id="httpwwwcfarumdeduferopticalsmoothing4html"><a href="http://www.cfar.umd.edu/~fer/optical/smoothing4.html">http://www.cfar.umd.edu/~fer/optical/smoothing4.html</a></h3>
<p>★★☆☆☆</p>
<p>This page explains the wavy grid illusion. Not directly useful but certainly interesting.</p>
<h3 id="httpwwwcvrlorg"><a href="http://www.cvrl.org/">http://www.cvrl.org/</a></h3>
<p>★☆☆☆☆</p>
<p>Here is some sort of scientific database, and a few low-power illusion images.</p>
<h3 id="httpwwwhandprintcomlscvscolorhtml"><a href="http://www.handprint.com/LS/CVS/color.html">http://www.handprint.com/LS/CVS/color.html</a></h3>
<p>★☆☆☆☆</p>
<p>Not super relevant but a degree’s worth of information about color and human perception. Maybe the authors would be of assistance.</p>
<h3 id="httpillusionoftheyearcom"><a href="http://illusionoftheyear.com/">http://illusionoftheyear.com/</a></h3>
<p>★★★★★</p>
<p>This contest collects and shares a large number of images, 20 this year and they’ve been at it for 12 years. 240 Images is pretty significant, but that’s just the top rated. The almost certainly have a huge stash of submissions, now just to get in conact with them and see if they are willing to share.</p>
<h3 id="httpwwwcogsciucieduddhoffillusionshtml"><a href="http://www.cogsci.uci.edu/~ddhoff/illusions.html">http://www.cogsci.uci.edu/~ddhoff/illusions.html</a></h3>
<p>★★★☆☆</p>
<p>This is 30ish images, at least half of which are static.</p>
<h3 id="httpmathworldwolframcomtopicsillusionshtml"><a href="http://mathworld.wolfram.com/topics/Illusions.html">http://mathworld.wolfram.com/topics/Illusions.html</a></h3>
<p>★★★☆☆</p>
<p>These are probably worth downloading.</p>
<h3 id="httpwwwsandlotsciencecom"><a href="http://www.sandlotscience.com/">http://www.sandlotscience.com/</a></h3>
<p>★★☆☆☆</p>
<p>These folks are selling a book that claims to have “200 optical illusions.” I doubt they would be willing to just hand them all over, but it’s worth a shot. It’s not like having their images in a zip file out there for deep learning is going to hurt their sales.</p>
<h3 id="httpviperlibyorkacuk"><a href="http://viperlib.york.ac.uk/">http://viperlib.york.ac.uk/</a></h3>
<p>★★★★★</p>
<p>Oh shoot, they have 1,860 images on file. I’ll have to look through and see what is usable for this research, but that is more than all of the other sources combined</p>
<h3 id="httpswwwredditcomropticalillusionstopsorttoptall"><a href="https://www.reddit.com/r/opticalillusions/top/?sort=top&t=all">https://www.reddit.com/r/opticalillusions/top/?sort=top&t=all</a></h3>
<p>★★★★☆</p>
<p>I’ve scraped reddit before for my meme-deepfryer based on cyclegan, so this should be a pretty easy way to collect many high quality images. It goes on for at least a few hundred posts, so it is worth it. Quality is lower and there are many photographic images, which is different from other datasets.</p>
<p>It seems that we will be able to get at most 1,000 to 2,000 images, which is alright. I don’t think GANs will be able to do anything useful, but if I knew for sure I wouldn’t be doing this. This dataset should be extremely valuable and development will continue full throttle.</p>
<h2 id="notes-on-image-content">Notes on Image Content</h2>
<p>Many images have logos or text. This might confuse the GAN, like how cats from the GAN paper had white impact font with illegible but English looking text over them due to the large amount of impact font English text in the dataset. Thankfully, the images don’t seem to have watermarks, which would not only confuse the network but would make the illusions less powerful. Most images are non-photographic, and maybe 10% of the reddit images are photographs. A great deal of sorting by hand could catagorize them, but I think using a deep learning technique would be sufficient. But that is for the next post about <strong>what to do with all this data</strong>.</p>I have been digging around for optical illusion datasets. They don’t seem to exist, so I will be laying the grounds for the creation of one and starting to collect data. I don’t know all that much about copyright, so if anything I describe sounds legally dubious I would really like to know.Adversarial Examples for human vision2018-01-09T00:00:00+00:002018-01-09T00:00:00+00:00https://robertmaxwilliams.github.io//Adversarial-Examples<p>Adversarial examples are very revealing about a neural net’s inner workings and weaknesses. <a href="https://blog.openai.com/adversarial-example-research/">This wonderful post</a> by open AI discusses the security implications of adversarial examples, and <a href="https://arxiv.org/abs/1712.09665">this arxiv paper</a> demonstrates extremely robust “adversarial patches” that can work on new networks that were not used in design. With adversarial example generation reaching this level of complexity, it raises the question of how immune the human vision system is to similar attacks, and what we can learn from attempting to generate adversarial examples for human vision.</p>
<h2 id="optical-illusions-as-adversarial-examples">Optical Illusions as Adversarial Examples</h2>
<p>Optical illusions are patterns that when observed by the human eye, create false impressions of non-existent stimulus. Examples of especially powerful illusions are <a href="http://www.michaelbach.de/ot/ang-SkyeGrating/index.html">Skye’s Oblique Grating</a> which causes straight line to appear parallel and the <a href="http://www.michaelbach.de/ot/lum-scGrid/index.html">Scintillating Grid</a> which causes black dots to appear anywhere you are not looking at directly. This can be seen as a related phenomenon to a misclassification by neural networks when observing an adversarial example. These patterns are painstakingly created by human artists, and developing a new kind of pattern (as opposed to a new instance of a known pattern) requires incredible skill and luck, especially given the large amount of existing patterns.</p>
<p><a href="https://en.wikipedia.org/wiki/Disruptive_coloration">Disruptive coloration</a> is another kind of optical illusion, but created by nature through evolution. Illusions of this type are more organic and generalize to nearly anything with a vision system, perhaps even to machine learning based systems. They are created through evolution with incredible amounts of trial and error on extremely complex environments and agents, on a scale not reproducible in simulation.</p>
<h2 id="generative-model-for-human-adversarial-examples">Generative Model for Human Adversarial Examples</h2>
<p><a href="http://research.nvidia.com/publication/2017-10_Progressive-Growing-of">Recent work</a> on generative adversarial networks (GANs) has shown that high resolution images of faces can be created using a large dataset of 30,000 images. This size and quality of images is not available for optical illusions; naively applying their methods would likely yield a model that is extremely overfit or generates nothing of value. Any attempt at pre-training on general images would also be fruitless, as optical illusions are usually non-photographic and exist outside the space of common visual stimulus. The number of static optical illusion images is likely in the low thousands, and the number of unique kinds of illusions is certainly very low, perhaps even less than one hundred. Creating a model capable of learning from such a small and limited dataset would represent a huge leap in generative models and understanding of human vision</p>
<h2 id="human-in-the-loop">Human in the Loop</h2>
<p>Both artistic designers of illusion images and the glacial process of evolution have access to active vision systems to verify their work against. An illusion artist can make an attempt at creating an illusion, observe its effect on their eyes, and add or remove elements to try to create a more powerful illusion. In an evolutionary process, every agent has a physical appearance and a vision system, allowing for patterns to be verified in their environment constantly. A GAN trained on existing illusions would have none of these advantages, and would be just as likely to create non-illusions as it is to create a novel class of images.</p>
<p>To improve the model beyond the existing data, its outputs can be classified by hand and fed back into the network. I am not sure if this form of dataset expansion has been used with generative models before.</p>
<h2 id="dataset">Dataset</h2>
<p>In coming blog posts, I will search out sources of illusion images and make further considerations for how to approach this problem.</p>Adversarial examples are very revealing about a neural net’s inner workings and weaknesses. This wonderful post by open AI discusses the security implications of adversarial examples, and this arxiv paper demonstrates extremely robust “adversarial patches” that can work on new networks that were not used in design. With adversarial example generation reaching this level of complexity, it raises the question of how immune the human vision system is to similar attacks, and what we can learn from attempting to generate adversarial examples for human vision.