Jekyll2023-10-03T00:21:56+00:00http://joshuacox.github.io/feed.xmlThe Blog of Joshua CoxWhere I blog about Jekyll, techromancy, docker, and other usually linux related stuffJoshua CoxWipe it with a cloth2022-06-17T00:00:00+00:002022-06-17T00:00:00+00:00http://joshuacox.github.io/blog/2022/06/17/wipe-it-with-a-cloth<p><img src="/img/thumbs/cloth_or_something3.jpg" alt="" /></p>
<p><a href="https://www.bleachbit.org/cloth-or-something">Bleachbit</a> became famous when Hillary used it on her machines:</p>
<blockquote>
<p>August 2015 Hillary Clinton was asked, “Did you wipe your email server?”
and she evasively replied, “Like with a cloth or something?”
A year later we found out that “cloth” was BleachBit,
a software application that deletes information
“so even God can’t read it,” as Congressman Trey Gowdy announced August 2016.</p>
</blockquote>
<p>Now, you can download <a href="https://www.bleachbit.org/download">bleachbit</a> for free, but we can also do the same thing given some of the basic tools available in every linux distro. Let’s explore some of this.</p>
<h2 id="identify-your-disks">Identify your disks</h2>
<p>First let’s identify the disk in question. If you have the convenience of having already popped the drive in question out of whatever machine it was in,
then let’s connect the device to our local machine for erasure using a device like <a href="https://amzn.to/3xYCcDS">this</a>.</p>
<p>Before we do connect we can take a snapshot of our current drive state with blkid like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo blkid | sort > /tmp/blkid1
</code></pre></div></div>
<p>I added the sort in there if anything gets out of order.
Then you can connect the device and capture a second time then diff it against the first one:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo blkid | sort > /tmp/blkid2
diff /tmp/blkid1 /tmp/blkid2
</code></pre></div></div>
<p>e.g.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>diff sort1 sort2
31a32
> /dev/sdd1: LABEL_FATBOOT="XZ" LABEL="XZ" UUID="81B7-D64B" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="d6969665-01"
</code></pre></div></div>
<p>In this case the drive in question is <code class="language-plaintext highlighter-rouge">/dev/sdd</code>.</p>
<p>You will, also, see entries in <code class="language-plaintext highlighter-rouge">dmesg</code> relating to the drive in question.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ sudo dmesg|grep -A18 'new high-speed'
[47775.756895] usb 1-11.2: new high-speed USB device number 12 using xhci_hcd
[47775.897140] usb 1-11.2: New USB device found, idVendor=abcd, idProduct=1234, bcdDevice= 1.00
[47775.897144] usb 1-11.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[47775.897145] usb 1-11.2: Product: UDisk
[47775.897146] usb 1-11.2: Manufacturer: General
[47775.897147] usb 1-11.2: SerialNumber: Љ
[47775.905101] usb-storage 1-11.2:1.0: USB Mass Storage device detected
[47775.905170] scsi host6: usb-storage 1-11.2:1.0
[47775.905204] usbcore: registered new interface driver usb-storage
[47775.907564] usbcore: registered new interface driver uas
[47776.917294] scsi 6:0:0:0: Direct-Access General UDisk 5.00 PQ: 0 ANSI: 2
[47776.917493] sd 6:0:0:0: Attached scsi generic sg3 type 0
[47776.917687] sd 6:0:0:0: [sdd] 7866368 512-byte logical blocks: (4.03 GB/3.75 GiB)
[47776.917807] sd 6:0:0:0: [sdd] Write Protect is off
[47776.917809] sd 6:0:0:0: [sdd] Mode Sense: 0b 00 00 08
[47776.917932] sd 6:0:0:0: [sdd] No Caching mode page found
[47776.917934] sd 6:0:0:0: [sdd] Assuming drive cache: write through
[47776.919350] sdd: sdd1
[47776.919934] sd 6:0:0:0: [sdd] Attached SCSI removable disk
</code></pre></div></div>
<p>We can use a few linux utilities to get more info on the device:</p>
<h4 id="fdisk">fdisk</h4>
<p>With fdisk the <code class="language-plaintext highlighter-rouge">Disk model:</code> line can be very useful in identifying a disk.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ sudo fdisk -l /dev/sdd
Disk /dev/sdd: 3.75 GiB, 4027580416 bytes, 7866368 sectors
Disk model: UDisk
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xd6969665
Device Boot Start End Sectors Size Id Type
/dev/sdd1 2048 7866367 7864320 3.8G 83 Linux
</code></pre></div></div>
<h4 id="blkid">blkid</h4>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ sudo blkid /dev/sdd*
/dev/sdd: PTUUID="d6969665" PTTYPE="dos"
/dev/sdd1: LABEL_FATBOOT="XZ" LABEL="XZ" UUID="81B7-D64B" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="d6969665-01"
</code></pre></div></div>
<h4 id="lsblkd">lsblkd</h4>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ sudo lsblk /dev/sdd
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sdd 8:48 1 3.8G 0 disk
└─sdd1 8:49 1 3.8G 0 part
</code></pre></div></div>
<p>Now, if you’re dealing with a remote machine you won’t be able to easily plug the device in like this and we can’t use diff easily to identify the device.
So invoking those commands without a specifier will show all the attached disks on the system.
(I, also, include parted here and not above because <code class="language-plaintext highlighter-rouge">parted -l /dev/sdd</code> ignores the specifier and returns all disks anyway.)</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ sudo fdisk -l
$ sudo blkid
$ sudo lsblk
$ sudo parted -l
</code></pre></div></div>
<h3 id="dd">dd</h3>
<p>Ok, now that we have identified the disk let’s erase it. The steps above are very important as these next commands are very dangerous and will happily erase your running system if you ask it to, so double check and make sure we have the correct disk in question.</p>
<p>The disk we identified above was <code class="language-plaintext highlighter-rouge">/dev/sdd</code> but might also look like <code class="language-plaintext highlighter-rouge">/dev/hdb</code>, or <code class="language-plaintext highlighter-rouge">/dev/vda</code>, or <code class="language-plaintext highlighter-rouge">/dev/nvme0n1p8</code> depending on your system.</p>
<p>Some of the main options we’ll be using are:</p>
<ol>
<li><code class="language-plaintext highlighter-rouge">bs</code> for block size, <code class="language-plaintext highlighter-rouge">count</code> for how many blocks to write</li>
<li><code class="language-plaintext highlighter-rouge">if</code> which is in-file or our source</li>
<li><code class="language-plaintext highlighter-rouge">of</code> which is out-file or our target.</li>
</ol>
<p>I am going to be using a low <code class="language-plaintext highlighter-rouge">count</code> on the below commands so that they don’t take that long on this usb drive I am using as an example. To zero and/or random the entire disk leave out the <code class="language-plaintext highlighter-rouge">count</code> option entirely and the <code class="language-plaintext highlighter-rouge">dd</code> command will continue writing to the end of the disk.</p>
<h4 id="devzero">/dev/zero</h4>
<p><code class="language-plaintext highlighter-rouge">/dev/zero</code> is a special ‘device’ in <code class="language-plaintext highlighter-rouge">/dev</code> that returns an unlimited amount of zeros.</p>
<p>The first method we’ll try is a quick zero:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ sudo dd if=/dev/zero bs=4M count=32 of=/dev/sdd
32+0 records in
32+0 records out
134217728 bytes (134 MB, 128 MiB) copied, 82.2589 s, 1.6 MB/s
</code></pre></div></div>
<p>That will write 128M of zeros to the first sectors of the disk <code class="language-plaintext highlighter-rouge">/dev/sdd</code>, note that for the most part this destroys without the possibility of anyone retrieving it, there is a lengthy discussion <a href="https://superuser.com/questions/1202267/securely-wiping-a-drive-with-dd-dev-zero-or-dev-urandom">here</a> and I’m certain elsewhere.o</p>
<p>But let’s explore further…</p>
<h4 id="devrandom">/dev/random</h4>
<p><code class="language-plaintext highlighter-rouge">/dev/random</code> is a special ‘device’ in <code class="language-plaintext highlighter-rouge">/dev</code> that returns an unlimited amount of pseudo random data, as it is ‘pseudo’ random it should not be used where real randomness is required. But for the purposes of obfuscating what was previously on the disk this is will make it very difficult to retrieve data from magnetic media* (*solid state media is a different story).</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ time sudo dd if=/dev/random bs=4M count=1 of=/dev/sdd
1+0 records in
1+0 records out
4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.282267 s, 14.9 MB/s
</code></pre></div></div>
<h4 id="devurandom">/dev/urandom</h4>
<p><code class="language-plaintext highlighter-rouge">/dev/urandom</code> is a special ‘device’ in <code class="language-plaintext highlighter-rouge">/dev</code> that returns an unlimited amount of random data (that has a higher quality of randomness than <code class="language-plaintext highlighter-rouge">/dev/random</code>, of note, it is made by a computer so pure randomness is up for debate).</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>time sudo dd if=/dev/urandom bs=4M count=1 of=/dev/sdd
1+0 records in
1+0 records out
4194304 bytes (4.2 MB, 4.0 MiB) copied, 2.62567 s, 1.6 MB/s
</code></pre></div></div>
<p>All of these will write 128M of random to the first sectors of the disk <code class="language-plaintext highlighter-rouge">/dev/sdd</code></p>
<h3 id="haveged">Haveged</h3>
<p>Of note, you can install <a href="https://www.issihosts.com/haveged/">haveged</a>,
and <code class="language-plaintext highlighter-rouge">/dev/urandom</code> will constantly be filled with high quality rando
and anything that relies on it will become significantly faster,
with our above dd command ran again after starting haveged we reduces 2.6s to 0.3s.
That’s nigh on a full order of magnitude!</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo systemctl start haveged
time sudo dd if=/dev/urandom bs=4M count=1 of=/dev/sdd
1+0 records in
1+0 records out
4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.301117 s, 13.9 MB/s
sudo dd if=/dev/urandom bs=4M count=1 of=/dev/sdd 0.01s user 0.01s system 8% cpu 0.316 total
</code></pre></div></div>
<h3 id="luks">Luks</h3>
<p>We can also just use cryptography itself to fill the disk with high quality random data.</p>
<p>First we will use the <code class="language-plaintext highlighter-rouge">cryptsetup luksFormat</code> command on <code class="language-plaintext highlighter-rouge">/dev/sdd</code>, at the first prompt you must type YES in all caps, then you are prompted for a password twice, and the disk will be formatted for encryption:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo cryptsetup luksFormat /dev/sdd
WARNING!
========
This will overwrite data on /dev/sdd irrevocably.
Are you sure? (Type 'yes' in capital letters): YES
Enter passphrase for /dev/sdd:
Verify passphrase:
sudo cryptsetup luksFormat /dev/sdd 25.82s user 0.97s system 79% cpu 33.708 total
</code></pre></div></div>
<p>Then open it, you will be prompted for the password you just gave above:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo cryptsetup open /dev/sdd wipecrypt
Enter passphrase for /dev/sdd:
</code></pre></div></div>
<p>create a filesystem on the open encrypted device:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ sudo mkfs.ext4 -L wipecrypt /dev/mapper/wipecrypt
mke2fs 1.46.5 (30-Dec-2021)
Creating filesystem with 979200 4k blocks and 244800 inodes
Filesystem UUID: 53aa7b9f-4dec-42fa-9ebc-7da1fcd3ddf3
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736
Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information:
</code></pre></div></div>
<p>Create a mnt spot for it, mount it, and then create a zero file to echo some zeros into:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ sudo mkdir /mnt/wipecrypt && sudo mount /dev/mapper/wipecrypt /mnt/wipecrypt && sudo touch /mnt/wipecrypt/zero && sudo chown $USER: /mnt/wipecrypt/zero
</code></pre></div></div>
<p>Now let’s write some zeros:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ /usr/bin/time dd if=/dev/zero of=/mnt/wipecrypt/zero bs=4M count=10
10+0 records in
10+0 records out
41943040 bytes (42 MB, 40 MiB) copied, 0.0188705 s, 2.2 GB/s
0.00user 0.02system 0:00.02elapsed 100%CPU (0avgtext+0avgdata 6388maxresident)k
0inputs+81920outputs (0major+1162minor)pagefaults 0swaps
</code></pre></div></div>
<p>Here is a fancy version that uses the yes command to write zeros instead of dd and /dev/zero, this is actually faster than dd, even split across two processes like above, though the usb drive is very slow and unlikely to show much difference here:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>time yes 0 |pv > /mnt/wipecrypt/zero
^C1.80GiB 0:06:29 [4.73MiB/s] [ <=> ]
yes 0 0.00s user 0.86s system 0% cpu 6:29.61 total
pv > /mnt/wipecrypt/zero 0.03s user 0.83s system 0% cpu 6:29.61 total]
ls -alh /mnt/wipecrypt/zero
Permissions Size User Date Modified Name
.rw-r--r-- 1.9G thoth 17 Jun 12:57 /mnt/wipecrypt/zero
</code></pre></div></div>
<p>Which while 4.73MiB/s is slower than the 2.2GB/s above this was sustained over six minutes, whereas the dd command was over a very short period of time.</p>
<p>If you just want to securely delete a single file you can use other commands like <a href="https://linux.die.net/man/1/shred">shred</a>, but for a whole blockdevice I prefer one of the above methods.</p>Joshua CoxBlog Like A Hacker with Tmuxinator + SpaceVIM + Jekyll2022-05-18T00:00:00+00:002022-05-18T00:00:00+00:00http://joshuacox.github.io/jekyll/2022/05/18/Tmuxinator-VIM-Jekyll-Blog-Like-A-Hacker<p><img src="/img/vim_drill_small.jpeg" alt="" /></p>
<p>This is an ode to Tom Preston Werner’s incecption <a href="http://tom.preston-werner.com/2008/11/17/blogging-like-a-hacker.html">post</a> to <a href="https://jekyllrb.com">Jekyll</a>.
While I am very impressed with what <a href="https://prose.io/">prose.io</a> is doing for people who want ‘simple’, I on the other hand have put in my time
steep learning curves of vim, bash, git, screen, tmux, byobu are behind me. I have these skills so I want to use them, right?
I don’t have time to click buttons and wait on submit times, my text editor is more than an extension of my body it exudes my essence
and is intrinsically part of my personality. Ok, enough ranting I should have driven off all but the resilient. Let’s get started.</p>
<p>If you are here then I most likely either don’t need to tell you the virtues of VIM or you’re an EMACS fan looking to troll.
In which case, I have one thing to tell you, <code class="language-plaintext highlighter-rouge">EMACS is a fantastic Operating System, that is lacking a decent text editor</code>, zing!
And we are gonna flat out ignore people who pay for their text editor. Ok, everyone gone?</p>
<p>Let me hook the rest of you up with the secret to destroy all other editors and IDEs:</p>
<h2 id="spacevim">SpaceVIM</h2>
<p><img src="/img/thumbs/spacevim.png" alt="" /></p>
<p><a href="http://spacevim.org/">spacevim.org/</a></p>
<p>You can install SpaceVIM with a curl oneliner::</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl -sLf https://spacevim.org/install.sh | bash
</code></pre></div></div>
<p>You will then have the most advanced hacker text editor on the planet. See the <a href="https://spacevim.org/documentation/">docs</a>.</p>
<h2 id="tmux">Tmux</h2>
<p>Now let’s use it like a proper hacker, and that is inside a <a href="https://github.com/tmux/tmux/wiki">Tmux</a> session.</p>
<p>Now, to be fair, we should probably mention <a href="https://linuxize.com/post/how-to-use-linux-screen/">screen</a>, which is how people used to multiplex terminal sessions, until tmux came along. Which about a decade ago eclipsed screen in development momentum, even this <a href="https://superuser.com/a/236160/291115">comment</a> has a decade of edits! Go judge for yourself though, the source code for <a href="https://github.com/tmux/tmux">tmux</a> and <a href="https://git.savannah.gnu.org/cgit/screen.git">screen</a> are there for your inspection.</p>
<p>Funny story, I used to be a big screen user over a decade ago, and I started using <a href="https://linuxize.com/post/how-to-use-linux-screen/">byobu</a>, and one day on my system I realized I didn’t even have screen installed and yet byobu worked just fine. Long story short I found out I was actually running tmux and not screen in the background, and I had no clue! Suffice it to say because of byobu I’ve been using tmux for over a decade using screen’s escape sequence <code class="language-plaintext highlighter-rouge">ctrl-a</code>, and all seems to be good.</p>
<p>This is because byobu wraps tmux very well and changes the leader back to <code class="language-plaintext highlighter-rouge">ctrl-a</code> (where tmux defaults to <code class="language-plaintext highlighter-rouge">ctrl-b</code>), there are noted other differences like <code class="language-plaintext highlighter-rouge">ctrl-b x</code> kills a session in tmux, but byobu remaps this back to the much more sane and harder to hit by mistake <code class="language-plaintext highlighter-rouge">ctrl-a k</code>.</p>
<p>Which brings me to the great ruby project <a href="https://github.com/tmuxinator/tmuxinator">Tmuxinator</a> to manage and create tmux sessions easily using yaml files!</p>
<p>After installing <a href="https://github.com/tmuxinator/tmuxinator">Tmuxinator</a>, let’s start by making a new tmuxinator session called “blog” –> <code class="language-plaintext highlighter-rouge">tmuxinator new blog</code></p>
<p>Which will create a new file –> <code class="language-plaintext highlighter-rouge">~/.config/tmuxinator/blog.yml</code>, and then open it up in your chosen <code class="language-plaintext highlighter-rouge">$EDITOR</code>, which whould be set to <code class="language-plaintext highlighter-rouge">vim</code> in order to use our new spacevim setup.</p>
<p>To utilize byobu, I set the <code class="language-plaintext highlighter-rouge">tmux_command</code> with:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>tmux_command: byobu
</code></pre></div></div>
<p>This starts my sessions with <code class="language-plaintext highlighter-rouge">byobu</code>, instead of <code class="language-plaintext highlighter-rouge">tmux</code>.</p>
<p>Here is a bare minimum example that I am using with this blog:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>name: blog
root: ~/git/joshuacox.github.io/
tmux_command: byobu
windows:
- jekyll:
layout: main-vertical
panes:
- vim _drafts/*
- jekyll s --drafts
</code></pre></div></div>
<p>So long as you have the shell <a href="https://github.com/tmuxinator/tmuxinator#completion">completion</a>, then you can now use the short hand:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>mux start blog
</code></pre></div></div>
<p>And you’ll have a custom tmux session dedicated to your blog.</p>
<p>Now let’s experiment with your new pimped out spacevim editor, first I recommend you turn on the <a href="https://spacevim.org/layers/lang/html/">html layer</a>, the <a href="https://spacevim.org/layers/lang/javascript/">javascript layer</a>, and the <a href="https://spacevim.org/layers/lang/markdown/">markdown layer</a>, you can see all the <a href="https://spacevim.org/layers/">available layers here</a>.</p>
<p>And the general Spacevim <a href="https://spacevim.org/documentation/">docs</a>.</p>Joshua CoxCyanogenmod Amazon Kindle Fire 20152017-03-01T00:00:00+00:002017-03-01T00:00:00+00:00http://joshuacox.github.io/blog/2017/03/01/Cyanogenmod-Amazon-Kindle-Fire-2015<p><a href="http://forum.xda-developers.com/amazon-fire/orig-development/root-t3272362">root</a></p>
<p><a href="http://flashfireapk.com/">flashfire</a></p>
<p>Further information should be found here :
<a href="http://wiki.cyanogenmod.org/w/Unofficial_Ports#Fire_Tablet">Cyanogenmod Device unofficial wiki list link</a>
Unless it becomes an official port, which we sincerely hope it does</p>
<p>XDA dev thread for <a href="http://forum.xda-developers.com/amazon-fire/orig-development/rom-cm-12-1-2015-11-15-t3249416">CM 12.1 for Amazon Fire Kindle 2015</a></p>
<p><a href="http://forum.xda-developers.com/amazon-fire/development/amazon-fire-5th-gen-supertool-root-t3272695">supertool</a></p>
<p><a href="http://www.droidviews.com/get-cyanogenmod-12-1-for-amazon-fire-7-2015-stable/">how-to from droidnews</a></p>
<p><a href="http://forum.xda-developers.com/amazon-fire/general/fireos-5-1-1-attempted-downgrade-t3263960/page9">bricked?</a></p>Joshua CoxrootRepackt2017-02-22T00:00:00+00:002017-02-22T00:00:00+00:00http://joshuacox.github.io/blog/2017/02/22/repackt<p><img src="/img/packt.png" alt="" /></p>
<p>Repackt - The
<a href="https://www.packtpub.com/">Packtpub</a>
epub file renamer</p>
<p>You can get a free book daily from
<a href="https://www.packtpub.com/">Packtpub</a>
<a href="https://www.packtpub.com/packt/offers/free-learning">here</a></p>
<p>The problem is that they name the file with a strange number (most
likely an inventory index or similar)</p>
<p>So I created this script to rename the files so that they are easier to
find in a folder full of similarly named epubs. It works by zgrep’ing
for the title field in the <code class="language-plaintext highlighter-rouge">.ocf</code> file</p>
<h3 id="usage">Usage</h3>
<h5 id="list">list</h5>
<p>repackt list $STRING - show all files that have $STRING in their name</p>
<h5 id="dry">dry</h5>
<p>repackt dry $STRING - dry run show the moves that would take place for all files that have $STRING in their name</p>
<h5 id="epub">epub</h5>
<p>repackt epub $STRING - rename all files that have $STRING in their name</p>
<h3 id="example-usage">Example Usage</h3>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>repackt epub 978
</code></pre></div></div>
<p>this will move a file called 978132434345.epub to TheProperTitle.epub and any other files which match the $STRING</p>
<h5 id="list-1">list</h5>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>➜ Backt ./repackt list 978
-rw-r--r-- 1 user user 8.6M Feb 20 22:27 97812345678.epub
-rw-r--r-- 1 user user 583K Feb 20 22:27 97812345678_code.zip
</code></pre></div></div>
<h5 id="dry-1">dry</h5>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>➜ Backt ./repackt dry 978
mv 97812345678.epub MasteringLinuxShellScripting.epub
mv 97812345678_code.zip MasteringLinuxShellScripting.code.zip
</code></pre></div></div>
<h5 id="epub-1">epub</h5>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>➜ Backt ./repackt epub 978
-rw-r--r-- 1 user user 8.6M Feb 20 22:27 97812345678.epub
mv 97812345678.epub MasteringLinuxShellScripting.epub
mv 97812345678_code.zip MasteringLinuxShellScripting.code.zip
-rw-r--r-- 1 user user 583K Feb 20 22:27 MasteringLinuxShellScripting.code.zip
-rw-r--r-- 1 user user 8.6M Feb 20 22:27 MasteringLinuxShellScripting.epub
</code></pre></div></div>
<h3 id="todo">Todo</h3>
<p>In the future I’d like it to convert pdfs and mobi files too, but for
now I only download the epubs as they seem to be the smallest in size
most of the time.</p>
<h3 id="follow">Follow</h3>
<p>You can follow this project <a href="https://joshuacox.github.io/repackt/">here</a></p>Joshua CoxBruteForceDomainSearch2017-01-20T00:00:00+00:002017-01-20T00:00:00+00:00http://joshuacox.github.io/blog/2017/01/20/BruteForceDomainSearch<p>Brute forcing your next domain name</p>
<p><a href="http://thalhalla.github.io/BruteForceDomainSearch/">Brute Force Domain Search</a></p>
<ul>
<li>Is a bit of perl to help you find open domain names that are available
for purchase.</li>
</ul>
<p>Anytime you go to starting a business or pretty much any other entity,
the first thing you ask yourself is what to name it. And this more
often than not leads to the question ‘Is this name available as a
domain?’…</p>
<p>At which point someone inevitably looks up the domain in question and
says “It’s already taken”, and the search continues.</p>
<p>To alleviate this problem I thought I’d invert the process and brute
force determine which domains are available.</p>
<p>Of note this is “brute force” and is likely to get you banned ten ways from Sunday! Do not ever run this script as it will destroy your computer and set fire to whatever dwelling you currently inhabit!
You’ve been warned (I’ve added some throttling and a block of public DNS servers but this is still a brute force script)</p>
<h3 id="example-usage">Example Usage</h3>
<p>Given you have perl properly installed (and it exists at
<code class="language-plaintext highlighter-rouge">/usr/bin/perl</code>, and you are not on arch linux where everything is too
new, then you you can do this</p>
<p><code class="language-plaintext highlighter-rouge">bruteforcedomainsearch.pl [--startingNumber 6 --finishingNumber 8 --throttle 10 --sleepthrottle 5 --tld '.com' -v]</code></p>
<p>Not much to see here, does what it says on the tin , e.g.
that will hunt down all available 6, 7, and 8 letter domains available
and limit you to ten requests with each resolver before sleeping for five seconds (roundabouts it was a simple throttle nothing fancy)
it will start dumping out available domain names into STDOUT (at <code class="language-plaintext highlighter-rouge">-v</code> it will only print out errors and available domain names) and all successfull domain names will be recorded in /tmp/domlog</p>
<h4 id="detailed-eplanations-of-the-arguments">Detailed Eplanations of the Arguments</h4>
<p><code class="language-plaintext highlighter-rouge">--startingNumber</code></p>
<ul>
<li>The number of letters we will start iterating from (i.e. 1 = a.com, 5 = aaaaa.com)</li>
</ul>
<p><code class="language-plaintext highlighter-rouge">--finishingNumber</code></p>
<ul>
<li>The number of letters we will finish iterating at (i.e. 1 = a.com, 5 = aaaaa.com)</li>
</ul>
<p><code class="language-plaintext highlighter-rouge">--throttle</code></p>
<ul>
<li>This is the number of loops to hit the Nameservers with before sleeping</li>
</ul>
<p><code class="language-plaintext highlighter-rouge">--sleepthrottle</code></p>
<ul>
<li>This is the number of seconds to sleep once we hit the throttle</li>
</ul>
<p><code class="language-plaintext highlighter-rouge">--forks</code></p>
<ul>
<li>This is the number of parallel processes to fork</li>
</ul>
<p><code class="language-plaintext highlighter-rouge">--tld</code></p>
<ul>
<li>This is the Top Level Domain (e.g. ‘.com’) to hit the Nameservers with before sleeping</li>
</ul>
<p><code class="language-plaintext highlighter-rouge">-v</code></p>
<ul>
<li>verbosity (notice you can stack more v’s at the end to get more verbosity)</li>
</ul>
<p><code class="language-plaintext highlighter-rouge">-vvvvvvvvv</code></p>
<ul>
<li>Really verbose</li>
</ul>
<h3 id="makefile">Makefile</h3>
<p>I use the <code class="language-plaintext highlighter-rouge">Makefile</code> to keep my usage a bit more consistent</p>
<p><code class="language-plaintext highlighter-rouge">make full</code></p>
<ul>
<li>full rundown from 1 to 10 letters</li>
</ul>
<p><code class="language-plaintext highlighter-rouge">make xi</code></p>
<ul>
<li>will do the same as above and install any dependencies using <code class="language-plaintext highlighter-rouge">lib::xi</code></li>
</ul>
<p><code class="language-plaintext highlighter-rouge">make reqs</code></p>
<ul>
<li>will install cpanminus and then <code class="language-plaintext highlighter-rouge">lib::xi</code> in debian</li>
</ul>
<h3 id="docker">Docker</h3>
<p>The <code class="language-plaintext highlighter-rouge">Makefile</code> will also net you some docker actions as well</p>
<p>Most of this section works by virtue of the <code class="language-plaintext highlighter-rouge">--cidfile="cid"</code> option to
the <code class="language-plaintext highlighter-rouge">docker run</code> command</p>
<p><code class="language-plaintext highlighter-rouge">tmp/domlog</code></p>
<ul>
<li>Using the Makefile to create the docker container will result in a <code class="language-plaintext highlighter-rouge">tmp</code> directory here, which will contain the resulting <code class="language-plaintext highlighter-rouge">/tmp/domlog</code> from inside the container, so no need to go into the container to retrieve it</li>
</ul>
<p><code class="language-plaintext highlighter-rouge">make a</code></p>
<ul>
<li>A macro which builds a docker image locally, runs it and displays the logs</li>
</ul>
<p><code class="language-plaintext highlighter-rouge">make build</code></p>
<ul>
<li>builds a docker image locally</li>
</ul>
<p><code class="language-plaintext highlighter-rouge">make log</code></p>
<ul>
<li>shows logs of the running docker</li>
</ul>
<p><code class="language-plaintext highlighter-rouge">make run</code></p>
<ul>
<li>runs a docker image locally</li>
</ul>
<p><code class="language-plaintext highlighter-rouge">make clean</code></p>
<ul>
<li>kills and removes the docker container</li>
</ul>
<h4 id="support">Support</h4>
<p>For the latest useage, see the github page:
<a href="http://thalhalla.github.io/BruteForceDomainSearch/">Brute Force Domain Search</a></p>
<p>For questions, or other help, please post an <a href="http://thalhalla.github.io/BruteForceDomainSearch/issues">issue</a></p>Joshua CoxBrute forcing your next domain nameConvert Wordpress to static PDQ2016-03-26T00:00:00+00:002016-03-26T00:00:00+00:00http://joshuacox.github.io/blog/2016/03/26/Convert-Wordpress-to-static-PDQ<p>Ok, today let’s kill off a wordpress site before it gets hacked and leave up a static unhackable version of the site,
note that using this method you can archive any publicly available site, be it wordpress, joomla, drupal, some proprietary
CMS that you have grown to hate, whatever, whenever.</p>
<p>There are a few ways to do this, the simplest being <code class="language-plaintext highlighter-rouge">wget -m</code>, e.g. let’s ‘mirror’ example.com</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> wget -m example.com
</code></pre></div></div>
<p>You now have a folder <code class="language-plaintext highlighter-rouge">example.com</code> with a static version of your site that is nigh invulnerable to hackers
and you can leave up for years without worrying about updating or maintenance.</p>
<h4 id="templatize">Templatize</h4>
<p>Now that we have a static version how do we make changes without having to rewrite every page? We convert the static site into templates.
If you’ll look at example.com/index.html you’ll notice we already have the base structure for our site.</p>
<p>So let’s fork this starter project:</p>
<p><a href="http://smartergiving.org/free-nonprofit-starter-website/">http://smartergiving.org/free-nonprofit-starter-website/</a></p>
<p>go here to it’s github home page:</p>
<p><a href="https://github.com/smartergiving/free-nonprofit-starter-website">https://github.com/smartergiving/free-nonprofit-starter-website</a></p>
<p>click ‘fork’ in the upper right hand corner (you’ll need a free github account for this)</p>
<p>then you’ll notice this file:</p>
<p><a href="https://github.com/smartergiving/free-nonprofit-starter-website/blob/master/_layouts/default.html">https://github.com/smartergiving/free-nonprofit-starter-website/blob/master/_layouts/default.html</a></p>
<p>where you’ll see statements like this:</p>
<p>{% include head.html %}</p>
<p>which points to a file called <code class="language-plaintext highlighter-rouge">head.html</code> in the <code class="language-plaintext highlighter-rouge">_includes</code> directory here:</p>
<p><a href="https://github.com/smartergiving/free-nonprofit-starter-website/tree/master/_includes">https://github.com/smartergiving/free-nonprofit-starter-website/tree/master/_includes</a></p>
<p>Alter that file to include the ‘head’ part of your theme (navbar and any other elements up top before the body)</p>
<p>Then do the same for the footer:</p>
<p><a href="https://github.com/smartergiving/free-nonprofit-starter-website/blob/master/_includes/footer.html">https://github.com/smartergiving/free-nonprofit-starter-website/blob/master/_includes/footer.html</a></p>
<p>And you are done with theming! It could not be simpler. YMMV, of course you might have issues
getting all of your elements (CSS, images, galleries, etc), but removing the complexity of a wordpress theme is
a smart idea. If only because a good portion of the wordpress attacks I’ve seen relied upon known vulnerabilities
introduced by very popular <a href="https://wpvulndb.com/themes">themes</a>.<br />
Of all programmers, themers seem to be the least aware of security issues.</p>
<h4 id="migrate-old-content">Migrate old content</h4>
<p>It may be easier to just migrate your content with one of the many migration utilities here:</p>
<p><a href="https://import.jekyllrb.com/docs/home/">https://import.jekyllrb.com/docs/home/</a></p>
<p>You’ll notice on the right many links to “importers” including wordpress:</p>
<p><a href="https://import.jekyllrb.com/docs/home/">https://import.jekyllrb.com/docs/home/</a></p>
<p>As stated on that page: This importer only converts your posts and creates YAML front-matter. It does not import any layouts, styling, or external files (images, CSS, etc.).</p>
<p>So you’ll still need to import your theme as above</p>
<h5 id="why">Why?</h5>
<p>I’m going to go on record here and state, IF you are not a PHP programmer, and you are not a MySQL database admin,
and you do not have the money to hire someone full-time to be those positions, then you have no business having a
website made with PHP and MySQL. This includes wordpress, joomla, drupal, and just about any CMS you can think of.
This stuff is not trivial, and security updates should be taken seriously. It is much to your advantage to eliminate
your exposure to programming languages if you are not a progammer.</p>
<p>So do yourself a favor and hire someone to create a static site for you generated from templates (like Jekyll or Ghost),
and you will not have to deal with hackers taking advantage of your site.
You’ll not only spend less money and time initially,
the savings over the long term will be substantial</p>
<h4 id="dynamic-elements">Dynamic elements</h4>
<p>Ok, what does a person need on their website that dynamic things like
PHP and MySQL provide? And do we have some alternatives?</p>
<h6 id="gui-editor--check">GUI editor? Check!</h6>
<p><a href="http://prose.io/">http://prose.io/</a></p>
<p>Also in the future we might see WYSIWYG support like:</p>
<p><a href="http://substance.io/">http://substance.io/</a></p>
<p>see these posts:</p>
<p><a href="https://github.com/prose/prose/issues/139">https://github.com/prose/prose/issues/139</a>
and
<a href="https://github.com/prose/prose/issues/143">https://github.com/prose/prose/issues/143</a></p>
<h6 id="commenting--check">Commenting? Check!</h6>
<p>discource (open source DIY solution):</p>
<p><a href="http://www.discourse.org/">http://www.discourse.org/</a></p>
<p>or disqus:</p>
<p><a href="http://joshuacox.github.io/jekyll/2015/11/27/disqus-and-jekyll/">http://joshuacox.github.io/jekyll/2015/11/27/disqus-and-jekyll/</a></p>
<p>or facebook:</p>
<p><a href="http://joshuacox.github.io/jekyll/2015/11/28/facebook-comments-and-jekyll/">http://joshuacox.github.io/jekyll/2015/11/28/facebook-comments-and-jekyll/</a></p>
<p>Of the three I can recommend both discource and disqus, facebook I found to be problematic on nearly half the browsers
I tested on, and since disqus will let you login with your facebook account and not vice versa disqus wins out in more people
being able to comment on your content, while blocking spammer, hackers, etc,
and they give you options for monetization, so I consider this one of the few
proprietary services to be truly beneficial for the small time blogger.</p>
<h6 id="sharing-with-social-media-check">Sharing with social media? Check!</h6>
<p><a href="http://joshuacox.github.io/jekyll/2015/12/01/Super-Simple-Sharing-Buttons/">http://joshuacox.github.io/jekyll/2015/12/01/Super-Simple-Sharing-Buttons/</a></p>
<h6 id="links-images-headings-etc-check">Links, images, headings, etc? Check!</h6>
<p><a href="http://joshuacox.github.io/jekyll/2015/12/02/0-to-60-in-Markdown/">http://joshuacox.github.io/jekyll/2015/12/02/0-to-60-in-Markdown/</a></p>
<p>Notice any of this can be done on ANY static template generator site,
this is not exclusive to jekyll. I just use it here as an example.</p>
<h6 id="e-commerce-check">E-commerce? Check!</h6>
<p>Jekyll-Store:</p>
<p><a href="http://www.jekyll-store.com/">http://www.jekyll-store.com/</a></p>
<p>or with amazon:</p>
<p><a href="http://joshuacox.github.io/amazon/">http://joshuacox.github.io/amazon/</a></p>
<h6 id="forms-check">Forms? Check!</h6>
<p>formkeep:</p>
<p><a href="https://formkeep.com/guides/how-to-make-a-contact-form-in-jekyll?utm_source=github&utm_medium=jekyll-docs&utm_campaign=contact-form-jekyll">https://formkeep.com/guides/how-to-make-a-contact-form-in-jekyll?utm_source=github&utm_medium=jekyll-docs&utm_campaign=contact-form-jekyll</a></p>
<p>simpleform:</p>
<p><a href="https://getsimpleform.com/">https://getsimpleform.com/</a></p>
<p><a href="http://sebastien.saunier.me/blog/2014/04/15/you-do-not-need-a-database-for-your-contact-form.html">http://sebastien.saunier.me/blog/2014/04/15/you-do-not-need-a-database-for-your-contact-form.html</a></p>
<p>who recommends formspree:</p>
<p><a href="https://formspree.io/">https://formspree.io/</a></p>
<p>and wufoo:</p>
<p><a href="http://www.wufoo.com/">http://www.wufoo.com/</a></p>
<p>stackoverflow question:</p>
<p><a href="http://stackoverflow.com/questions/17983972/github-contact-form-with-jekyll">http://stackoverflow.com/questions/17983972/github-contact-form-with-jekyll</a></p>
<p><a href="https://coderwall.com/p/8lq1ba/how-to-create-a-contact-form-for-a-github-pages-served-jekyll-website">https://coderwall.com/p/8lq1ba/how-to-create-a-contact-form-for-a-github-pages-served-jekyll-website</a></p>
<h3 id="jekyll-and-the-future">Jekyll and the future</h3>
<p>You may know that:</p>
<p><a href="https://www.healthcare.gov/">https://www.healthcare.gov/</a></p>
<p>was built using jekyll. But this is ancient history now, the leader of jekyll has left to do
other things and why continue to use Jekyll? Because the principles
are sound, you can build a site in Jekyll today and it can stay up for
years without maintenance or updates.</p>
<p>So whereto next?</p>
<p>There are tons of great projects springing up based on these principles.</p>
<p>“But it’s static so that means it’s hard”, bullshit. It’s easier than
dealing with a wordpress theme, and you can be up and rolling in three
minutes if you follow this:</p>
<p><a href="http://jekyllbootstrap.com/">http://jekyllbootstrap.com/</a></p>
<p>and that guy went on to build ruhoh:</p>
<p><a href="http://ruhoh.com/">http://ruhoh.com/</a></p>
<p>which is very friendly and easy to use.</p>
<p>or you can use ghost:</p>
<p><a href="https://ghost.org/">https://ghost.org/</a></p>
<p>there are hundreds of new platforms out there now. And this does not
mean you cannot have dynamic things in them, as I’ve already shown all
the normal elements like commenting can be had easily.</p>
<p>If you need something custom, fire up a nodejs server, or sinatra, or
ruby on rails, and you can do amazingly complex stuff like serve up a
MMORPG through browser experience.</p>
<p>You can even fire up a drupal instance and use it as a database
backend API for your static site.</p>
<p><a href="https://groups.drupal.org/headless-drupal">https://groups.drupal.org/headless-drupal</a></p>
<p>which is how I envision some really cool stuff happening with D3 as a visualizer</p>
<p><a href="https://d3js.org/">https://d3js.org/</a></p>
<p>So the future is rich with options, get rid of wordpress now, stick to simple templates that generate static content
and pull in the dynamic things you need using javascript and inhouse solutions or use many of the great truly free
to use services like disqus and let them handle the hackers so you can get back to creating content.</p>Joshua CoxOk, today let’s kill off a wordpress site before it gets hacked and leave up a static unhackable version of the site, note that using this method you can archive any publicly available site, be it wordpress, joomla, drupal, some proprietary CMS that you have grown to hate, whatever, whenever.aria2c2016-02-11T00:00:00+00:002016-02-11T00:00:00+00:00http://joshuacox.github.io/blog/2016/02/11/aria2c<h4 id="download-from-all-the-sources-at-once">Download from all the sources at once</h4>
<p><a href="https://aria2.github.io/">aria2</a> is one of those old tools that has been around forever but you forget about it.
It will download one file from many sources at once, saturating your download enabling you to take full advantage
of whatever connection you are paying for. Of note, this sort of behavior is likely to give all other users in your
house issues while you are consuming the entire connection bandwidth. But if you’re paying for the connection,
why wait around on ISOz to DL?
Well I fired it up again to fight this awful century unlink connection,
there are many useful examples on their <a href="https://aria2.github.io/">homepage</a>
but let me give you a quick one that is missing from that list that names the output file
(useful for those messy links from sourceforge), so as an example to download debian kde 8.3 for amd64</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>aria2c -x2 -o debian-8.3.0-amd64-kde-CD-1.iso http://ftp.riken.jp/Linux/debian/debian-cd/8.3.0/amd64/iso-cd/debian-8.3.0-amd64-kde-CD-1.iso http://mirror.steadfast.net/debian-cd/8.3.0/amd64/iso-cd/debian-8.3.0-amd64-kde-CD-1.iso http://mirrors.ocf.berkeley.edu/debian-cd/8.3.0/amd64/iso-cd/debian-8.3.0-amd64-kde-CD-1.iso http://mirrors.bloomu.edu/debian-cd/8.3.0/amd64/iso-cd/debian-8.3.0-amd64-kde-CD-1.iso http://mirrors.advancedhosters.com/debian-cd/8.3.0/amd64/iso-cd/debian-8.3.0-amd64-kde-CD-1.iso
</code></pre></div></div>
<p>Note the related projects and UI frontends associated with this project, especially powerpill</p>
<h5>Related Projects</h5>
<ul>
<li><a href="https://github.com/tatsuhiro-t/apt-metalink"><strong>apt-metalink</strong></a>: Faster package downloads for Debian/Ubuntu</li>
<li><a href="http://xyne.archlinux.ca/projects/powerpill/"><strong>powerpill</strong></a>: Pacman wrapper for parallel and segmented downloads.</li>
<li><a href="http://xyne.archlinux.ca/projects/python3-aria2jsonrpc/"><strong>python3-aria2jsonrpc</strong></a>: A wrapper class around Aria2’s JSON RPC interface.</li>
</ul>
<h5>UI Frontends</h5>
<ul>
<li><a href="https://github.com/ziahamza/webui-aria2"><strong>webui-aria2</strong></a>: Web browser interface for aria2 (2012 GSOC project)</li>
<li><a href="http://uget.visuex.com/"><strong>uGet</strong></a>: the Linux Download Manager</li>
</ul>Joshua CoxDownload from all the sources at onceText Editors And Code2016-01-29T00:00:00+00:002016-01-29T00:00:00+00:00http://joshuacox.github.io/blog/2016/01/29/Text-Editors-And-Code<p>One universal question I can ask any programmer is, what text editor do you use?
Easily the most important tool in the programmers toolkit is the text
editor. Nothing will make you faster and more powerful than knowing
how to quickly and efficiently edit text.</p>
<h3 id="vim">VIM</h3>
<p>My favorite editor is VIM, it is very old, archaic,
has a very steep learning curve,
but is extremely rewarding in terms of efficiency once
you utilize its hands on keyboard nature.
In that vein I have a site for you:</p>
<p><a href="http://vim-adventures.com/">http://vim-adventures.com/</a></p>
<p>also</p>
<p><a href="http://www.openvim.com/">http://www.openvim.com/</a></p>
<p>lazy and just want to watch a video?</p>
<p><a href="http://derekwyatt.org/vim/tutorials/">http://derekwyatt.org/vim/tutorials/</a></p>
<p>and</p>
<p><a href="http://vimcasts.org/">http://vimcasts.org/</a></p>
<p>Also, don’t forget to try out simply <code class="language-plaintext highlighter-rouge">vimtutor</code> it is usually installed alongside vim in most distributions</p>
<h4 id="pimp-my-vim">Pimp my VIM</h4>
<p>While this may be too much for you at the moment, I want to show endgame.
Here is how I have my vim configured, it is super pimped out with text
prediction, surround replacement alteration macros, advanced movement
(so you go anywhere without using a mouse), debugging without leaving
the text editor, everything you’d see in a full featured IDE
(Integrated Development Environment), but using vim.</p>
<p><a href="http://vim.spf13.com/">http://vim.spf13.com/</a></p>
<h3 id="emacs-vs-vim">EMACS vs VIM</h3>
<p>Also, to round you out I’ll introduce the competition. Emacs vs. Vim
is a religious debate, there is no clear winner unless you are a
devotee to one of them already ( I am a VIM racist ). But I will say
that Emacs can be just as powerful as vim. Here is some humor:
<a href="https://en.wikipedia.org/wiki/Editor_war#Humor">https://en.wikipedia.org/wiki/Editor_war#Humor</a></p>
<p>my favorite is this one</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Emacs is "a great operating system, lacking only a decent editor"
</code></pre></div></div>
<h3 id="ides">IDEs</h3>
<p>Wanting to try out a giant IDE? Xcode on mac is not bad, you can also
download the android studio for any platform:</p>
<p><a href="http://developer.android.com/sdk/index.html">http://developer.android.com/sdk/index.html</a></p>
<p>it includes a pimped out copy of IntelliJ which is a nice environment:</p>
<p><a href="https://plugins.jetbrains.com/plugin/?idea&pluginId=631">https://plugins.jetbrains.com/plugin/?idea&pluginId=631</a></p>
<p>and android studio used to be based on eclipse:</p>
<p><a href="https://eclipse.org/downloads/">https://eclipse.org/downloads/</a></p>
<p>which is also free.</p>
<p>Notice you can get eclipse to behave like vim:
<a href="http://vrapper.sourceforge.net/home/">http://vrapper.sourceforge.net/home/</a></p>
<p>and vim to behave like eclipse:
<a href="http://eclim.org/">http://eclim.org/</a></p>
<p>Try them all out
and get familiar with the differences, but be sure and do it low tech
style as well so you know the meat of what’s going on (python given as an example)</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code> vim helloworld.py
</code></pre></div></div>
<p>and write at least one script that way (hint: to run it afterwards just
do <code class="language-plaintext highlighter-rouge">python helloworld.py</code>), you don’t need all that GUI stuff, and it can
actually hinder you much like crutches must be taken away at some
point before the person can ‘run’.</p>
<h2 id="git">GIT</h2>
<p>Do you get git? If your text editor is one of the most important
tools of programming, then how you save your text files is just as
important. Git ‘saves’ a snapshot of a directory, meaning that it
‘saves’ each and every file in a directory at a point in time.</p>
<p>I have a tutorial here:
<a href="http://joshuacox.github.io/cli-Intro-to-Git/">http://joshuacox.github.io/cli-Intro-to-Git/</a></p>
<p>more resources:
<a href="https://try.github.io/levels/1/challenges/1">https://try.github.io/levels/1/challenges/1</a></p>
<p><a href="https://www.codecademy.com/learn/learn-git">https://www.codecademy.com/learn/learn-git</a></p>
<p><a href="https://help.github.com/articles/good-resources-for-learning-git-and-github/">https://help.github.com/articles/good-resources-for-learning-git-and-github/</a></p>
<p>as always let me know when you have questions, send code my way, I’ll
help debug!</p>
<p>Hint: a good way to send me code is to put up a repo in github and send me a link, and I’ll send you pull requests of changes.</p>Joshua CoxOne universal question I can ask any programmer is, what text editor do you use? Easily the most important tool in the programmers toolkit is the text editor. Nothing will make you faster and more powerful than knowing how to quickly and efficiently edit text.Persistent Icinga2 Docker on RaspberryPi Cluster2016-01-05T00:00:00+00:002016-01-05T00:00:00+00:00http://joshuacox.github.io/monitoring/2016/01/05/Persistent-Icinga2-Docker<p><img src="/img/icinga_logo.jpg" alt="" /></p>
<p><a href="https://www.icinga.org/icinga/icinga-2/">Icinga2</a> is a next gen fork of <a href="https://www.nagios.com/">Nagios</a>.
While using the open source base from the organization <a href="https://www.nagios.org/">Nagios</a> you always felt like you were missing out
by not paying your dues to Nagios Enterprise. Along came Icinga, a fork with much of the features offered by the enterprise version of nagios
for free and much more. It was open source and all the plugins still worked. So repositories of generic plugins like this <a href="https://www.monitoring-plugins.org/">one</a>
did not go to waste. But with the second version they really upped the ante, the full feature list is <a href="https://www.icinga.org/icinga/icinga-2/features/">here</a>.<br />
Some of the most impressive improvements are in the clustering area. Well that makes it a great candidate for dockerization.</p>
<p>Earlier we used ansible to spin up a docker container I have a previous <a href="http://joshuacox.github.io/docker/2015/11/28/Icinga2-Docker/">post</a>
on this topic, but I felt the other methods were failing me and I wanted more persistence</p>
<p>I started trying to convert the official container into a persistent instance with <a href="https://github.com/joshuacox/mkicinga">mkicinga</a>,
however, mysql was acting funny on me, and I’d rather not have it in the image anyhow, so I went back to the fork of jeyk’s old debian based <a href="http://joshuacox.github.io/docker-icinga2">Dockerfile</a>
I peeled mysql out into it’s own conatiner and then fiddled with the initdocker script from the official repo until it started working in debian</p>
<h4 id="requirements">Requirements</h4>
<ul>
<li><a href="http://astore.amazon.com/joshuacox-20/detail/B00MV6TAJI/189-3117811-4891539">RaspberryPi Full kit</a> at least one, but I’d recommend three</li>
<li><a href="http://joshuacox.github.io/docker/2015/12/13/RaspberryPi-Docker-Cluster-Consul-Swarm/">Cluster setup</a> from our previous post there</li>
</ul>
<h4 id="usage">Usage</h4>
<p>should be easy first pull the temporary recipe up</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>make temp
</code></pre></div></div>
<p>Let it finish populating the databases, (you can watch by using <code class="language-plaintext highlighter-rouge">make logs</code> ctrl-c to exit viewing the logs, despite what the warning says you will actually not kill the container
[because your are killing the <code class="language-plaintext highlighter-rouge">tail -f</code> of the log watching process not the process itself in this case])
and <code class="language-plaintext highlighter-rouge">killall mysql</code></p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>make enter # we have ‘entered’ the container
exit # back in the host environment now
</code></pre></div></div>
<p>then grab all the persistent volumes</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>make grab
</code></pre></div></div>
<p>then run in <code class="language-plaintext highlighter-rouge">prod</code> mode, take this ‘prod’ with a grain of salt, I’ll leave it to you to understand all security implications of the
MYSQL_ROOT_PASS file hanging out after these Makefiles have ran (i.e. on all docker machines you should implicitly trust all users with the ‘docker’ group as they effectively have root on the machine)</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>make prod
</code></pre></div></div>
<h3 id="icinga-web-2">Icinga Web 2</h3>
<p>Icinga Web 2 can be accessed at http://localhost:3080/icingaweb2 w/ <code class="language-plaintext highlighter-rouge">icingaadmin:icinga</code> as credentials.</p>
<p>The configuration is located in /etc/icingaweb2 which is exposed as <a href="#volumes">volume</a> from
docker.</p>
<p>By default the icingaweb2 database is created including the <code class="language-plaintext highlighter-rouge">icingaadmin</code> user. Additional
configuration is also added to skip the setup wizard.</p>
<h2 id="ports">Ports</h2>
<p>The following ports are exposed:</p>
<table>
<thead>
<tr>
<th>Port</th>
<th>Service</th>
</tr>
</thead>
<tbody>
<tr>
<td>22</td>
<td>SSH</td>
</tr>
<tr>
<td>80</td>
<td>HTTP</td>
</tr>
<tr>
<td>443</td>
<td>HTTPS</td>
</tr>
<tr>
<td>3306</td>
<td>MySQL</td>
</tr>
<tr>
<td>5665</td>
<td>Icinga 2 API & Cluster</td>
</tr>
</tbody>
</table>
<h2 id="volumes">Volumes</h2>
<p>These volumes can be mounted in order to test and develop various stuff.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>/etc/icinga2
/etc/icingaweb2
/var/lib/icinga2
/usr/share/icingaweb2
/var/lib/mysql
</code></pre></div></div>
<h4 id="clustering">Clustering</h4>
<h5 id="master-setup">master setup</h5>
<p>Now let’s set it up our master, enter our</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>make enter # we have ‘entered’ the container
icinga2 node wizard
</code></pre></div></div>
<p>you’ll answer ‘n’ to the first question as this is the master, when it asks you about bind hosts and ports just hit return the defaults are fine here.</p>
<h5 id="client-setup">client setup</h5>
<p>Now setup a seond host with icinga2 on it, you can use our docker container again if you like.
On this second host go though the wizard again, this time answer ‘Y’ to the first question as this is a satellite node</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>icinga2 node wizard
</code></pre></div></div>
<p>you will be prompted for a pki key at one point. It provides a hint, something similar to:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>icinga2 pki ticket --cn $HOSTNAME
</code></pre></div></div>
<p>you need to execute this on the master, it will output a key that you need to then paste into the prompt from the wizard back on the satellite node.<br />
This will setup encrypted communication between the master and satellite node.</p>
<p>Now it’s time to restart the nodes, if using my docker container just use the prod recipe again</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>make prod
</code></pre></div></div>
<h6 id="discovery-of-the-new-nodes-and-services">Discovery of the new nodes and services</h6>
<p>To see your new node log back into the master and issue this command:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>icinga2 node list
</code></pre></div></div>
<p>And then to add the configs incant this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>icinga2 node update-config
</code></pre></div></div>
<p>restart the nodes and you should have your new host auto discovered and monitored</p>
<p>For more details view the full <a href="http://docs.icinga.org/icinga2/latest/doc/module/icinga2/chapter/icinga2-client">Icinga2 client docs</a></p>
<h4 id="monitoring-plugins">Monitoring Plugins</h4>
<p>You might notice some plugins missing in the form of error messages of icinga searching for stuff in <code class="language-plaintext highlighter-rouge">/usr/lib64/nagios/plugins</code></p>
<p>CentOS and RHEL systems in particular only come with a dismal amount of plugins, so let’s install the gold standard in terms of plugins
download <a href="https://www.monitoring-plugins.org/download.html">here</a>, and then uncompress and install them</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gzip -dc monitoring-plugins-2.x.tar.gz | tar -xf -
cd monitoring-plugins-2.x
./configure
make
make install
</code></pre></div></div>
<p>Then change the <code class="language-plaintext highlighter-rouge">PluginDir</code> line in <code class="language-plaintext highlighter-rouge">/etc/icinga2/constants.conf</code></p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>const PluginDir = "/usr/local/nagios/plugins"
</code></pre></div></div>
<h3 id="distributed-monitoring-and-high-availability">Distributed Monitoring and High Availability</h3>
<p>Now to configure everything you’ll need to add each endpoint and zone as described here in the <a href="http://docs.icinga.org/icinga2/snapshot/doc/module/icinga2/chapter/distributed-monitoring-high-availability">Icinga2 docs</a></p>
<p>again main page here for my docker-icinga2:</p>
<p><a href="http://joshuacox.github.io/docker-icinga2">joshuacox.github.io/docker-icinga2</a></p>
<p>For more related errata visit my blog <a href="http://joshuacox.github.io/">joshuacox.github.io</a></p>Joshua CoxHappy Habitica2016-01-01T00:00:00+00:002016-01-01T00:00:00+00:00http://joshuacox.github.io/blog/2016/01/01/Happy-Habitica<p><img src="http://i1354.photobucket.com/albums/q698/thoth242/Wiki-wordmark_zpsid5nsoqx.png" alt="" /></p>
<p><img src="http://i1354.photobucket.com/albums/q698/thoth242/frabjabulous_zpsexkr3vkm.png" alt="" />
##Happy New Year folks!</p>
<p>Have you tried habitica? Your life has now been transformed into the latest form of augmented reality, you are now living the video game.
Earn XP and gold for doing things like cleaning the catbox, laundry, brushing your teeth, randomly find eggs from which you can hatch mammals like wolves.
But only if you find a bottle of nail polish first to color the animal with….. Or something like that.</p>
<p><img src="http://i1354.photobucket.com/albums/q698/thoth242/habitica-intro_zpsmngy1hhv.png" alt="" /></p>
<p>There have been a few of these things in the past. That is a todo list combined with an RPG.
what makes this one interesting is that they open sourced it</p>
<p><a href="https://github.com/HabitRPG/habitrpg">github.com/HabitRPG/habitrpg</a></p>
<p>And not just the web app, on their main page you can find the android application:</p>
<p><a href="https://github.com/HabitRPG/habitrpg-android">github.com/HabitRPG/habitrpg-android</a></p>
<p>the iOs app:</p>
<p><a href="https://github.com/HabitRPG/habitrpg-ios">github.com/HabitRPG/habitrpg-ios</a></p>
<p>And much, much more, including email servers, chrome extensions, more experimental phone <a href="https://github.com/HabitRPG/habitrpg-mobile">builds</a> that use angularJs and phonegap.</p>
<p>Kudos to them for open sourcing everything, I think I’ll join and actually buy from this company. The only thing that would make be feel better is if they were a <a href="http://www.webhosting.coop/">coop</a></p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/hgdeJnSili0" frameborder="0" allowfullscreen=""></iframe>Joshua Cox