<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Hacking-That-Camps-Yack on Amanda Regan</title><link>https://amanda-regan.com/tags/hacking-that-camps-yack/</link><description>Recent content in Hacking-That-Camps-Yack on Amanda Regan</description><generator>Hugo</generator><language>en-us</language><lastBuildDate>Wed, 26 Feb 2014 14:30:57 +0000</lastBuildDate><atom:link href="https://amanda-regan.com/tags/hacking-that-camps-yack/index.xml" rel="self" type="application/rss+xml"/><item><title>THATCamp Topic Modeling Results</title><link>https://amanda-regan.com/posts/thatcamp-topic-modeling-results/</link><pubDate>Wed, 26 Feb 2014 14:30:57 +0000</pubDate><guid>https://amanda-regan.com/posts/thatcamp-topic-modeling-results/</guid><description>&lt;p&gt;&lt;strong&gt;This post is the last in a set of 5 written by the &lt;a href="http://chnm.gmu.edu/courses/fellowship/?page_id=11"&gt;Digital History Fellows&lt;/a&gt; at the &lt;a href="http://chnm.gmu.edu/"&gt;Roy Rosenzweig Center for History and New Media&lt;/a&gt;. The original post, which was collectively written by the DH Fellows, can be found on our &lt;a href="http://chnm.gmu.edu/courses/fellowship/"&gt;Digital History Fellowship Blog&lt;/a&gt;.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;We have spent the last few weeks working to build a python script that would allow us to download and prep all of the THATCamp blog posts for topic modeling in MALLET (for those catching up, we detailed this process in a series of &lt;a href="http://chnm.gmu.edu/courses/fellowship/"&gt;previous posts&lt;/a&gt;). As our last post &lt;a href="http://chnm.gmu.edu/courses/fellowship/?p=448"&gt;detailed&lt;/a&gt;, we encountered a few more complications than expected due to foreign languages in the corpus of the text.  After some discussion, we worked through these issues and were able to add stoplists to the script for German, French, and Spanish.  Although this didn’t solve all of our issues and some terms do still show up (we didn’t realize there was Dutch too), it led to some interesting discussion about the methodology behind topic modeling.  Finally we were able to rerun the python script with the new stopwords and then feed this new data into MALLET.&lt;/p&gt;</description></item><item><title>Unexpected Challenges Result in Important and Informative Discussions: a transparent discussion about stripping content and stopwords</title><link>https://amanda-regan.com/posts/unexpected-challenges-result-important-informative-discussions-transparent-discussion-stripping-content-stopwords/</link><pubDate>Wed, 19 Feb 2014 14:00:07 +0000</pubDate><guid>https://amanda-regan.com/posts/unexpected-challenges-result-important-informative-discussions-transparent-discussion-stripping-content-stopwords/</guid><description>&lt;p&gt;&lt;strong&gt;This post is the fourth in a set of 5 written by the &lt;a href="http://chnm.gmu.edu/courses/fellowship/?pageid=11"&gt;Digital History Fellows&lt;/a&gt; at the &lt;a href="http://chnm.gmu.edu/"&gt;Roy Rosenzweig Center for History and New Media&lt;/a&gt;. The original post by Jannelle Legg can be found on the &lt;a href="http://chnm.gmu.edu/courses/fellowship/"&gt;Digital History Fellowship Blog&lt;/a&gt;.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;As described in previous &lt;a href="http://chnm.gmu.edu/courses/fellowship/?p=446"&gt;posts&lt;/a&gt;, the first year Digital Fellows at CHNM have been working on a project under the Research division that involves collecting, cleaning, and analyzing data from a corpus of THATCamp content. Having overcome the hurdles of writing some python script and using MySQL to grab content from tables in the backend of a Wordpress install, we moved on to the relatively straightforward process of running our stripped text files through MALLET.&lt;/p&gt;</description></item><item><title>Pre-Processing Text for MALLET</title><link>https://amanda-regan.com/posts/pre-processing-text-mallet/</link><pubDate>Tue, 18 Feb 2014 14:00:29 +0000</pubDate><guid>https://amanda-regan.com/posts/pre-processing-text-mallet/</guid><description>&lt;p&gt;&lt;strong&gt;This post is the third in a set of 5 written by the &lt;a href="http://chnm.gmu.edu/courses/fellowship/?page_id=11"&gt;Digital History Fellows&lt;/a&gt; at the &lt;a href="http://chnm.gmu.edu/"&gt;Roy Rosenzweig Center for History and New Media&lt;/a&gt;. The original post can be found on the &lt;a href="http://chnm.gmu.edu/courses/fellowship/"&gt;Digital History Fellowship Blog&lt;/a&gt;.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;In our previous &lt;a href="http://chnm.gmu.edu/courses/fellowship/?p=439"&gt;post&lt;/a&gt;, we described the process of writing a python script that pulled from the THATCamp MySQL Database. In this post, we will continue with this project and work to clean up the data we&amp;rsquo;ve collected and prepare it for some analysis. This process is known as &amp;ldquo;pre-processing&amp;rdquo;. After running our script in the THATCamp database all of the posts were collected and saved as text files. At this stage, the files are filled with extraneous information relating to the structure of the posts. Most of these are tags and metadata that would disrupt any attempts to look across the dataset. Our task here was to clean them up so they could be fed into MALLET. In order to do this, we needed to strip the html tags, remove punctuation, and remove common stopwords. To do this, we used chunks of code from the &lt;a href="http://programminghistorian.org/lessons/from-html-to-list-of-words-1"&gt;Programming Historian&amp;rsquo;s lesson on text analysis&lt;/a&gt; with python and modified the code to work with the files we had already downloaded.&lt;/p&gt;</description></item><item><title>Extracting Data from the THATCamp Database Using Python and MySQL</title><link>https://amanda-regan.com/posts/extracting-data-python-mysql/</link><pubDate>Sun, 09 Feb 2014 20:54:45 +0000</pubDate><guid>https://amanda-regan.com/posts/extracting-data-python-mysql/</guid><description>&lt;p&gt;&lt;strong&gt;This post is the second in a set of 4 written by the &lt;a href="http://chnm.gmu.edu/courses/fellowship/?page_id=11"&gt;Digital History Fellows&lt;/a&gt; at the &lt;a href="http://chnm.gmu.edu/"&gt;Roy Rosenzweig Center for History and New Media&lt;/a&gt;. The original post can be found on the &lt;a href="http://chnm.gmu.edu/courses/fellowship/"&gt;Digital History Fellowship Blog&lt;/a&gt;.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;This week we&amp;rsquo;ve continued to work on building a python script that will extract all of the blog posts from the various THATCamp websites. As &lt;a href="http://chnm.gmu.edu/courses/fellowship/?p=421"&gt;Jannelle described last week&lt;/a&gt;, our goal was to write a script that downloads the blog posts in plain text form and strips all of the html tags, stopwords, and punctuation so that we can feed it into MALLET for topic modeling and text analysis. After several long days and a lot of help from second year fellow Spencer Roberts, we&amp;rsquo;ve successfully gotten the code to work.&lt;/p&gt;</description></item><item><title>Spring Semester in Research and a THATCamp Challenge</title><link>https://amanda-regan.com/posts/spring-semester-research-thatcamp-challenge/</link><pubDate>Sun, 09 Feb 2014 19:39:04 +0000</pubDate><guid>https://amanda-regan.com/posts/spring-semester-research-thatcamp-challenge/</guid><description>&lt;p&gt;&lt;strong&gt;This post is the first in a set of 4 written by the &lt;a href="http://chnm.gmu.edu/courses/fellowship/?page_id=11"&gt;Digital History Fellows&lt;/a&gt; at the &lt;a href="http://chnm.gmu.edu/"&gt;Roy Rosenzweig Center for History and New Media&lt;/a&gt;. The original post by Jannelle Legg can be found on the &lt;a href="http://chnm.gmu.edu/courses/fellowship/"&gt;Digital History Fellowship Blog&lt;/a&gt;.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;The spring semester is here and the first year DH fellows have begun our rotation into the Research division of CHNM.&lt;/p&gt;
&lt;p&gt;To get the ball rolling, we spent a week working through the helpful tutorials at the&lt;a href="http://programminghistorian.org/"&gt;Programming Historian&lt;/a&gt;. As someone new to DH, with admittedly limited technical skill and knowledge, these were immeasurably useful. Each tutorial breaks content into smaller, less intimidating units. These can be completed in succession or selected for a particular topic or skill. While there is useful content for anyone, we focused our attention on &lt;a href="http://programminghistorian.org/lessons/introduction-and-installation"&gt;Python&lt;/a&gt; and &lt;a href="http://programminghistorian.org/lessons/topic-modeling-and-mallet"&gt;Topic Modeling&lt;/a&gt; with the aim of solving our own programming dilemma.&lt;/p&gt;</description></item></channel></rss>