Morning pages

Yesterday I spent most of my time trying to migrate a production WordPress site to my development environment. Normally, I’ve used Infinite WordPress’s site migration tools to move them, which does the trick of moving all the files and updating the database references to the site URL, but I don’t think it works when the site’s not public. I’m doing a lot of hacks with my Docker setup, importing databases, messing with file permissions, and duplicating a lot of my work since I like to sit downstairs at my desk during the day, and upstairs at night. So part of my challenge is trying to find a setup that works well for me.

I might have to make some sacrifices. JetBrains IDEs don’t like to work over the network, so I’ve got to add a directory sync if I want to keep the files on my network server and work from both workstations. At least I can run Docker from the remote machine, but it’s not supported by the IDE, so I’ll have to figure out how to fit that into my workflow.

The girls were good. Elder did everything I asked her, and got her extra screen time. She did Typing, piano, and two sessions of math in Khan’s, and we managed to keep the house tidy. So that’s a big parenting win. She’s already up and on her laptop right now, ostensibly doing typing, but I don’t hear too much of it going on over there. Maybe she’s doing math.

We had a bit of excitement yesterday when bitcoin went on a little bit of a tear. I noticed it shot up to touch ten thousand and got excited. Elder came over and said “it went UP,” excitedly. We watched the fight for a few minutes, before it dumped, and then went on a bike ride.

I moved my entire Ethereum stash over to BlockFi. There’s always a moment of horror after publishing a large transaction to the blockchain when the doubt sets in. Wondering whether the address was copied correctly or if my opsec failed and some hacker changed the receiving address while it was in my clipboard. Did I check the address. I usually do a small test transaction before sending over the big one, but it still makes me nervous. Especially after reading about the mining firm that said someone sent a $144 Eth transaction with $131 million in gas fees.

So now I have a fair chunk of my assets up on BlockFi. I haven’t touched but a fraction of my BTC; it’s just too much risk for me to do that. I’ve got a roughly even spit on there between BTC, ETH, and USD stablecoins, and I’m considering whether to put more USD there. I’ve still got the girl’s BTC accounts, but I don’t want to mix them with mine, and I’m not yet sure if I can open an account in their name or if I’ll have to do like I did for lending club and make multiple ones in my name.

Due to the coronavirus, the IRS is allowing 2019 IRA contributions up until July 1. I’m considering whether I want to do this, or throw some more cash into BlockFi. My IRA is on fire right now, I calculated 70% realized gains off of this market rally, and my unrealized gains for the year are much higher than when I calculated them a couple of weeks ago. I’ve still got active value average positions that are in play, and I’m probably going to be short on cash before they complete, so I need some powder. I just doing know whether I should sell some of my other positions, or put more cash into play. All of my current plays are under risk-adjusted position sizes, but my long term holdings are just sitting without any stops on them. With everyone going crazy on RobinHood these days, I should probably put some protections in place in case there’s another lockdown related pullback.

Yesterday, a client’s laptop failed, and I’m waiting on a vendor to go out there and swap a motherboard or something. The drive is encrypted, and while I’m certain I have the keys, I felt a shot of adrenaline course through my body when I remembered that I neglected to reinstall a backup program on her machine after replacing it. So I know what I’m doing today. What I don’t know is what I’m posting tomorrow for my newsletter. This post has been the type of rambling morning pages post that’s of no use to anyone but myself, and which is not the type of quality content that I want to be sending out to my LinkedIn network, or to the email list which I just salvaged from an old CSV file.

I’m going to let that one mull in my head today, and let it stew.

What is work?

two white rabbits

Down one rabbit hole after the other

I spent most of yesterday really digging into WordPress in a way that I really haven’t before: theme files. My current project has a customized version of the Twenty Seventeen theme, with lots of custom templates, fields, and functions that I need to move over to a new template. It’s taken me weeks to finally understand what the previous developer was doing, and there’s a fatal bug in the system somewhere that is deleting post data that I’m trying to uncover so I can clean things up. I figure my best course of action is to migrate everything to a staging site, start with a new theme, and start going through the plugins one by one to rebuild the content on the site. There are multiple pages and types of posts with custom fields that need to be displayed properly. I’m not really looking forward to having to debug someone else’s stylesheets, though.

Doing this kind of development isn’t ideal even on a staging site, given that the WordPress native code editor isn’t really suited to real work. I haven’t done PHP work in over ten years, but I downloaded PHPStorm and got started setting up a development environment. I was hoping to setup some sort of Git workflow for the site, but I didn’t find any options that were production ready, so I grabbed the files via FTP and quickly set to work.

WordPress has an official Docker image, so I set about configuring a Compse file for my local environment. There I ran into problems. I was trying to map my theme directory to the container’s, but I ran into issues with file permissions. I haven’t quite figured it out. I can change the permissions within the container to allow the container to use the files, but then they’re locked on my development host. So that’s my challenge for today, and one that will no doubt lead down many more rabbit holes.

This is just an example of the kind of stuff I do, that most people call work. Now this doesn’t have anything to do with my regular day job responsibilities, it’s for a client. And even if it wasn’t, it’s still the same type of activity that I would be doing for fun anyways. Although if you asked my wife if she thought I was having fun last night, she would have said that all the cursing and muttering I was doing under my breath would indicate otherwise. This particular project is a challenge for me because it involved a level of technical expertise that I don’t have, that I am forced to pick up in order to understand the issue — and hopefully solve it! It’s this area, right outside my current capabilities, that puts me in the zone and makes time fly.

It’s a drive that has gotten me where I am today, and has served me very well. Unfortunatley, it’s not something I find in my current day job, and is one of the main reasons why I’m looking else where these days. Part of the problem is the fact that the company constantly hovers on the edge of sustainabily and closure, but I have trouble reconciling that situation with my responsibility for it. Perhaps it’s that I don’t have any stake in the company, other than my current minimum viable salary. It’s allowed me to pursue other projects, including school and political activities, but has not offered anything for me in the way of growth in several years. I am not in sync with my boss in the way of the direction of the company or even the type of customers that we take on. The challenges are rote, and therefore not interesting to me. And they haven ‘t changed in years. Neither has my salary.

I’ve started reading Designing Your Life, by Bill Burnett and Dave Evans, and one of the first exercises that they ask readers to write a workfview reflection, defining how work relates to their life, money and others. This is my response to that, of course. Work has such a broad meaning to me. It’s not just your job, it’s also the things you do for your family and friends, chores around the house or the yard, spending time with family, and yes, helping your dad or whoever with their laptop from time to time. And one thing my dad taught me, that I’m trying to impress upon my girls, is that when there’s work to be done you just have to suck it up and do it.

Work is rewarding also, and can be fun. That’s not to say it can’t be repetitive or stressful,, the most panic-inducing heart attack moments I’ve had have been related to failures at work. But I’ve helped a lot of people, and it’s often fulfilling. That’s not to say that I haven’t had horrible, dirty jobs that I had to take because I was unemployed and living on couches, but most of them have been knowledge work, and pretty chill. These days it pays the bills, but it’s the work I do outside of work that is where I continue to learn and grow.

Hopefully my girls will be as lucky as I am, and be able to make a living doing what they love. Actually, it’s not luck, it’s by design. Obviously I am not where I want to be right now. Sure, my work life is probably better than ninety percent of the world’s population right now, and I have no room to complain about anything, but it’s it the human condition to want more, to want to be more? And to me, that’s what work is, the drive to improve, to become better. Constant improvement. Refine, iterate, repeat, repeat, repeat.

Leave it better than you found it

blue plastic trash bins on forest during daytime

Taking over abandoned or mismanaged projects

Taking over a project is a much different beast than starting from scratch. I think everyone knows that it’s easier starting from a blank slate most of the time. What separates the amateurs from the professionals though is the documentation that they leave for those that come after. Most of my recent work I’ve been the solo technical resource on a small team, and coming into a new project is often a mess, and trying to decipher someone else’s work without any form of documentation is a challenge. I make sure not to leave it that way.

I’ve been doing small business networks for well over a decade, and taking on a new client almost always starts with a network assessment, inventorying the equipment, and running some kind of network or system scanning tool to catch what else we might have caught and put it into a report. Internally, we’ve been using ITGlue to keep track of all our documentation, and it really comes in handy when handing off a client. In the past, taking over an account from another IT management firm has involved sitting down for an interview with the technical resources on the other team, making lots of notes, and then rebuilding the documentation in our system. And it usually involves some sort of roughly drawn up document with passwords and other critical information.

Lately, it seems we’ve been sending off runbooks left and right, containing all the documentation, checklists and SOPs that we’ve developed for a client. I remember the first time we handed off one to another firm, seeing the surprise in their eyes when we handed over a professional, looking document. It made a real good impression, and I almost considered jumping ship with them. That was over a year again, and here I am, finding myself going through the same situation again.

My recent consultation work has mostly involved taking over a stable of WordPress sites. I’ve been using WordPress for years for this blog and others, but I’ve usually kept things very simple. Just download a nice theme, and start writing. The sites I’ve been taking on recently are much more complicated. There’s usually two or three dozen plugins deployed, and some sort of complicated theme system in place that has some particular arcane way of adding a page or making changes to a header or footer.

Since my focus here is just about writing, I intentionally decided not to spend any time on presentation. I’m literally still using the default Twenty Seventeen theme that came with WordPress out of the box. I’ve looked at some premium themes for it to give it some zazz, but ultimately decided that the effort wasn’t a priority for me. Not so with the other projects. I’ve been using an Envato Elements subscription to source my themes and templates, but each one seems to carry it’s own set of required plugins and design methodology. Figuring out how to tweak them is its own challenge.

I recently took over a site for a client. It was mostly in good shape, but had been neglected for several years. I wanted to make some changes to it, but without understanding how everything was put together, it’s proven difficult. On top of that the original designer used a modified version of one of the default WordPress templates, so the choice was to start delving into the source code or start building from scratch. And again, there were about forty plugins being used, and I’m yet unaware of a simple way to trace an elment in a rendered WordPress site to its source. Plugins will often add elements to the Dashboard UI, or the document editor, and figuring out what goes with what is a slog.

So far, the choice for me has usually been to tear it all down and start from scratch. Cloning production to a staging site and deactivating all the plugins, to see what we’ve got, content wise, is usually the first step. Then I can all the pages and posts to see which elements are missing from the original site. “There’s a shortcode for a slideshow plugin, so let’s note that and re-enable that.” “Why is half of our content missing?” It’s because they used some post taxonomy plugin to put certain content in a separate blog. And so on and so on.

There’s one thing I picked up in the last year or so, called Architectural Design Records, or ADRs. It’s basically a decision making artifact that details the reasoning behind taking a particular design approach to something. They’re closely tied to user stories, and can be placed right in a Git repo with the rest of the source code. I’ve been trying to carry some of the ideas behind ADRs into my own projects, and not just software ones. It’s a good practice for any sort of system design, whether that is technical, business, or personal. Leaving these little artifacts behind for future you or for others seems that it can be a valuable practice, and will come in handy when the time come to tweak something months or years down the line. “Why on Earth did we decide to do x again? Oh yeah, here’s the ADR…”

I am defintely not a WordPress ninja. As the old saw goes, “the more you learn, the more you realize that you don’t know anything.” Give or take. Managing a web host reseller account and using a dedicated tool to keep WordPress installations and their related plugins up to date is simple enough, but taking over these sites and trying to redesign them with an eye toward user design, ecommerce and SEO is a completely different set of skills than I had hoped to working on. And I don’t know how far I want to develop it. Most likely I’ll be doing what I can to salvage the projects I’m working on and get them to the state where it can start generating some revenue, then I’ll start bringing in other resources to hand off tasks to. And when I do, I’ll have supporting documentation to hand off to them so that they can quickly get up to speed, a history of how things operate and why they were setup the way they are.

WordPress moban.html hack

So I just finished cleaning up one of the WordPress sites that I manage from a hack. I was checking Google Analytics and noticed a few irregularities. The first was a number of hits from China, and then noticed some URLs in the site description that didn’t belong there.

I checked the first URL and yep, we have a hack. I logged into the WP dashboard and immediately found two admin users. I did not find the urls in posts or pages, which was odd, so I started scrubbing the site. I found that I was locked out of several administrator functions, such as updating WordPress or installing new plugins. Thankfully, I was able to deploy them through Infinite WordPress. I ran several scans to check for modifications to the wp-admin directories, and even deleted them and uploaded them from a fresh download of WordPress over FTP.

I found several directories that were out of place: developerl, openbayl, and webstruct. The latter was filled with XML documents, some sort of sitemap:

<?xml version="1.0" encoding="UTF-8" ?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
     <url>
			 <loc>http://<my_hacked_site></loc> 
			 <lastmod>2020-04-07T10:13:06-05:00</lastmod> 
			 <changefreq>always</changefreq> 
			 <priority>1.0</priority> 
			 </url>
		     <url>
					 <loc>http://<my_hacked_site>/46/Auto-34cm-2-Meter-Kabel-C16620-Aerzetix-Radio/</loc> 
					 <lastmod>2020-04-07T10:13:06-05:00</lastmod>   
					 <changefreq>daily</changefreq> 
					 <priority>0.9</priority> 
					 </url>
				     <url>
					 <loc>http://<my_hacked_site>/141/35-mm-KlinkenStecker-und-Adapter-Handy-25-mm-Stecker-iPad-MP3Player/</loc> 
					 <lastmod>2020-04-07T10:13:06-05:00</lastmod>   
					 <changefreq>daily</changefreq> 
					 <priority>0.9</priority> 
					 </url>
				     <url>

The openbayl directory contained a moban.html file that contained some sort of HTML template, and the developerl directory seems to have the core part of the hack. It includes another moban.html file that contains a bastardized copy of the sites WordPress headers and footers, and some mangled content that appears to be scraped from the site as well. There was some sort of encoded key in a logs.txt file, and a map.log file pointing to the webstruct xml files. I’ve uploaded this file as a gist.

I downloaded a copy of these files, then deleted them from the site. After running security scans and looking for any additional files that didn’t belong, I was still locked out of installing plugins. I started pouring through the SQL data, looking for what happened. I checked the .htaccess file for any shenanigans, and disabled all plugins. My user appeared to have admin access in the database, and I verified that the administrator role had the install_plugin role in the wp_options wp_user_roles row. But when I added a PHP check in the site files, I didn’t have the role.

I had spent almost two hours digging through this. I had identified the time of the hack, and thankfully, I had a backup from the night before. I restored it, and functionality was back to normal.

I’m troubled that I don’t know how the hack occured. Everything was reasonably up to date. We were behind on a WordPress update, 5.3.x to 5.4, but I’m not aware of any vulnerabilities that would have allowed us to be hacked. Regardless, I took additional steps after restoring the site, including installing Sucuri Security and NinjaScanner. I’m also going to be deploying them on all sites under management.