Yesterday turned into a bit of a slog. I was so tired after writing yesterday’s entry that I went back to bed and managed to get in a powernap before I had to start the day in earnest. And I was so drained by the end of the day that I turned in an hour early. I don’t think I got much done. The new ASUS router came in the early afternoon, (LOVE IT), and I ran some errands around town, but I wasn’t especially productive. I made an attempt at writing today’s Substack, but I couldn’t make up my mind what to focus on. I started out writing about “the future of work”, but started diverging into a critique of the last forty years of economic and Fed monetary policy. There’s no doubt it will taking up much of my day.
Today is off to a much better start. The weather is beautiful. I watched a squirrel build a nest in a tree in the backyard, and it sounds like the juniper bush next to our deck has some bird hatchlings in it. And it’s Friday. The girls are eating breakfast, listening to Party in the USA for the third time in a row.
When I came down this morning and said “computer, good morning,” it reminded me that “today is Juneteenth, … a moment to reflect on the black experience in America.” It’s remarkable, since just yesterday we had a discussion during our Zombie, LLC scrum call about whether the company, of which we are a franchise, had made any statements in recognition of the BLM protests following George Floyd’s death. They had not. The conversation did not go well. The question came up from X., a young black woman, before Boss came on the line, and when I rephrased the question after he came on the line, his response was not great. It seemed to lack any empathy for X, as well as a general cluelessness about what was going on, and why. When I mentioned the responses from other firms that I had seen, he actually responded with, “well, let’s leave aside who’s paying for that for a moment.” I was so dumbfounded.
Part of my fatigue-induced procrastination involved catching up on some reading, namely Venkatesh Rao’s The Gervais Principle, which proposes a new model of interoffice relationships based off of The Office and the following image from Gaping Void. It’s blown my mind, really, and has given me some insight into why I’ve felt so unfulfilled at Zombie. I’m stuck in the overperforming loser category with no room for advancement. My boss is clueless. The only way for me to change that is to either fully become a slacker loser, or embrace my inner sociopath and get the hell out of doge. In a way, I’ve already decided to be a slacker, and focus my attention elsewhere.
I’m keeping today’s post short since I have more writing to do, and have already wasted enough of my morning dicking around with my new ASUS ZenWIFI. It’s already started malfunctioning, and I really didn’t want to futz with it. That said, it’s got a remarkable open interface, with SSH and telnet access to the underlying Linux OS. Asus has also put the full source code for the OS online, which is amazing for a corporate company. I’m disappointed that it’s already acting up and that I’m allowing it to distract me from my one task for today.
And last note: yesterday the girls and I setup my Mr. Beer homebrew keg. I should have my first batch available in four weeks.
This attack is not new, but the tactics are evolving, and some people are still behind the curve
I’ve been managing business networks for some time, and I’ve witnessed phishing attacks, where attackers attempt to steal a victim’s email login information, evolve the last few years. Yesterday I was alerted to a new variation on this traditional attack that I thought was worth sharing and dissecting, as you’ll see why.
Almost all of the attacks that I’ve seen stem from an email that a victim receives. Usually it’s someone that the victim has corresponded with in the past. The subject line and body vary, but there’s usually an external link where the victim is directed to in order to download some secure file. Normally, the victim arrives at a page that looks like a Google or Microsoft landing page, but of course they’re a fake, setup to steal the victim’s credentials.
If the phishers are successful, they’ll have gained access not only to the victim’s mailbox, but also any associated document storage systems like Google Drive, or Microsoft OneDrive or SharePoint. From there it’s all over, the attackers can download whatever they need, or if they discover that they’ve infiltrated a high value target, they might lurk, and prepare additional attacks.
In one particular case that I was involved in a few years ago, attackers managed to phish the CEO of a company. They discovered that they were going to be travelling from the East coast to the West, and waited until they were thirty thousand feet in the air to launch a fake CEO attack, requesting that their finance director wire tens of thousands of dollars to the perpetrators bank account as soon as possible. In this case, there were enough red flags that the attack was thwarted, but not before the attackers had used the CEOs mailbox to resend the phishing attack to everyone in their contact history.
And so the cycle repeats.
How not to be a victim
Normally, there are numerous red flags when phishing attempts happen, but it still surprises me the number of requests I get from people asking me to inspect an email for legitimacy.
Sometimes it’s as easy as examining the email recipient, or the actual link in the email, and finding that they don’t match. If Jane Doe’s corporate email is jdoe@corp.com, and you see your email client only displays “Jane Doe“, you might need to hover your mouse over it to see that the email is really from a different address altogether. (Hover over the link above to see what I’m talking about.) Most modern email clients have updated the way they display emails, making sure that the actual address is “Jane Doe <jdoe@corp.com>” or something similar.
However, there are still a number of businesses that haven’t taken precautions to protect their own email systems from being spoofed. That’s to say, there may not be anything stopping from someone from setting up a rogue email server and sending an email from anyone at that company. There are several methods to protect from this, known as SPF, DKIM and DMARC, that protect from this happening, so you may want to make sure that your domains are protected.
The flag that I look for is where the link is pointing. Just like email addresses, these URLs can be spoofed. Modern rich-text or HTML mail clients which allow special formatting can be used to try and trick users with links that misdirect users to hacked sites. So always check the URL. That official looking login page for your Office365 account might just be a fake sitting behind someone’s hacked WordPress site. CHECK. THE. URL.
These tips alone should prevent most people from falling victim to one of these attacks. If I’ve been drawn into investigating at this point, I usually go a step further and try to get the fake landing page taken down. Sometimes it’s easy to find the company who’s site has been hijacked, and usually a courtesy call is enough for me to consider my good deed done for the day. Sometimes the site is set up by the hackers themselves. A ten dollar web domain with a three dollar hosting account, paired with a free WordPress template is enough to start with. In these latter cases, I have to do a bit more work to find where the domain is registered and where the site is hosted. Then, an email to the company’s abuse department, and I’m done.
How you can stop it
And in almost every case that I’ve seen, it’s been a WordPress site that has been hosting the fake landing page. As it’s the software behind more than a third of all websites on the internet, it’s not surprising. But if you’ve got a business website running on WordPress and you’re not maintaining it or paying someone to manage it for you, then not only are you exposing yourself, your firm, and your clients to hacks, but you’re also partially responsible for any victims that fall prey through your site. Update your site, at least quarterly, or purchase a product or hire a firm that can check it on a regular basis for you.
How Chrome marked the site with the fake landing page. Firefox has no such warning.
Making sure email the security protocols mentioned earlier, (SPF, DKIM and DMARC) are enabled on your domains will prevent hackers from faking your domain and using it in an attack.
Using updated email software and security applications are also an effective way to mitigate these attacks. Make sure that your email client software is a recent version, or use a cloud-based one to make sure that you have access to the latest anti-phishing tools. And make sure you use them! It still astonishes me how many small firms haven’t enabled two factor authentication for their employees, or even looked at the protection services that are available from their email providers.
And one of the most important things you can do is train your staff how not to fall victim to these attacks. There are a number of firms that can deploy phishing attempts against your staff, and provide training to those who fail to avoid it.
Attackers upping their game
What concerned me with the attack I witnessed was the way that the attackers changed their tactics to evade some of the more advanced mitigation techniques that are in place to stop these cybercrimes. A number of enterprise level email security services have the ability to filter out these malicious links and block them from the recipient. They usually rely on some sort of whitelist or blacklist to allow certain domains through. In the case this week, the victim was sent to Live.com, which is Microsoft’s ID portal for Outlook.com and OneDrive accounts. To the casual observer, it looked like a legitimate OneNote notebook, and there was no breach at this point. No doubt most organization administrators would have no problem with users going there.
Of course within this OneNote page was the real trap, a link to the fake landing page. Thankfully the mark in this case, noting that the OneNote page was addressed from a person different than the original email, was suspicious enough not to fall for it. That said, when I was alerted to it and took a look at the OneNote page without the context of the original email, my initial thought was that it was legit. I almost cleared it! A second read turned up some irregular grammar, which is when I noticed the external link and the O365 landing page. Even then I still had to look up the domain registration on the site, two months earlier using an Asian registrar, before I was convinced it wasn’t some sort of Single Sign On configuration.
Technology changes fast, and cybersecurity is a cat and mouse game between attackers and the security professionals that protect your personal and business assets from these dangerous breaches. If you need help with managing your infrastructure or mitigation strategy against these attempts, let’s discuss it. Whether it’s email and network infrastructure, securing your website, or doing mock infiltration testing or employee training. I can help.
I spent most of yesterday really digging into WordPress in a way that I really haven’t before: theme files. My current project has a customized version of the Twenty Seventeen theme, with lots of custom templates, fields, and functions that I need to move over to a new template. It’s taken me weeks to finally understand what the previous developer was doing, and there’s a fatal bug in the system somewhere that is deleting post data that I’m trying to uncover so I can clean things up. I figure my best course of action is to migrate everything to a staging site, start with a new theme, and start going through the plugins one by one to rebuild the content on the site. There are multiple pages and types of posts with custom fields that need to be displayed properly. I’m not really looking forward to having to debug someone else’s stylesheets, though.
Doing this kind of development isn’t ideal even on a staging site, given that the WordPress native code editor isn’t really suited to real work. I haven’t done PHP work in over ten years, but I downloaded PHPStorm and got started setting up a development environment. I was hoping to setup some sort of Git workflow for the site, but I didn’t find any options that were production ready, so I grabbed the files via FTP and quickly set to work.
WordPress has an official Docker image, so I set about configuring a Compse file for my local environment. There I ran into problems. I was trying to map my theme directory to the container’s, but I ran into issues with file permissions. I haven’t quite figured it out. I can change the permissions within the container to allow the container to use the files, but then they’re locked on my development host. So that’s my challenge for today, and one that will no doubt lead down many more rabbit holes.
This is just an example of the kind of stuff I do, that most people call work. Now this doesn’t have anything to do with my regular day job responsibilities, it’s for a client. And even if it wasn’t, it’s still the same type of activity that I would be doing for fun anyways. Although if you asked my wife if she thought I was having fun last night, she would have said that all the cursing and muttering I was doing under my breath would indicate otherwise. This particular project is a challenge for me because it involved a level of technical expertise that I don’t have, that I am forced to pick up in order to understand the issue — and hopefully solve it! It’s this area, right outside my current capabilities, that puts me in the zone and makes time fly.
It’s a drive that has gotten me where I am today, and has served me very well. Unfortunatley, it’s not something I find in my current day job, and is one of the main reasons why I’m looking else where these days. Part of the problem is the fact that the company constantly hovers on the edge of sustainabily and closure, but I have trouble reconciling that situation with my responsibility for it. Perhaps it’s that I don’t have any stake in the company, other than my current minimum viable salary. It’s allowed me to pursue other projects, including school and political activities, but has not offered anything for me in the way of growth in several years. I am not in sync with my boss in the way of the direction of the company or even the type of customers that we take on. The challenges are rote, and therefore not interesting to me. And they haven ‘t changed in years. Neither has my salary.
I’ve started reading Designing Your Life, by Bill Burnett and Dave Evans, and one of the first exercises that they ask readers to write a workfview reflection, defining how work relates to their life, money and others. This is my response to that, of course. Work has such a broad meaning to me. It’s not just your job, it’s also the things you do for your family and friends, chores around the house or the yard, spending time with family, and yes, helping your dad or whoever with their laptop from time to time. And one thing my dad taught me, that I’m trying to impress upon my girls, is that when there’s work to be done you just have to suck it up and do it.
Work is rewarding also, and can be fun. That’s not to say it can’t be repetitive or stressful,, the most panic-inducing heart attack moments I’ve had have been related to failures at work. But I’ve helped a lot of people, and it’s often fulfilling. That’s not to say that I haven’t had horrible, dirty jobs that I had to take because I was unemployed and living on couches, but most of them have been knowledge work, and pretty chill. These days it pays the bills, but it’s the work I do outside of work that is where I continue to learn and grow.
Hopefully my girls will be as lucky as I am, and be able to make a living doing what they love. Actually, it’s not luck, it’s by design. Obviously I am not where I want to be right now. Sure, my work life is probably better than ninety percent of the world’s population right now, and I have no room to complain about anything, but it’s it the human condition to want more, to want to be more? And to me, that’s what work is, the drive to improve, to become better. Constant improvement. Refine, iterate, repeat, repeat, repeat.
It’s a beautiful Saturday morning here. I’m bringing the girls over to their grandmothers later today, which will mean the first time the house has been free of the kids in six weeks. I don’t even know what my wife and I will do with ourselves.
I’ve got one more task to finish college, a how to guide for faculty and students on how to use GitLab for note sharing. Should take me a couple hours of writing, tops. Then I’m done. I already got my grade for the group project class, an A, and I turned in my assignments and exam for my numerical methods class last night. That project is definitely going on my resume/portfolio site, and will probably get a full write up at some point. The only problem with it is that it can’t compile in CodeBlocks, so I’ll probably get 50% on it. I may muck around later and see if I can get it to compile via GitLab. The professor is likely grading on a super steep curve, so I shouldn’t really worry about it. There’s no doubt that I’ll pass, the question is whether I get a C or an A. After all the work I did compiling class materials into the GitLab wiki, I’ll be disappointed with anything less than a A.
However it goes, I should wind up with at least a 3.5 GPA. Six years of classes, part time, while holding down a job, raising two kids, and running two political campaigns. I sure am proud of myself. Now if I could just bring myself to take one of these $80,000 year jobs that I see listed on LinkedIn. I’m going to finish updating my resume, put it up on the new CV site that I built, and start applying to anything with the salary disclosed. We’ll see who bites. Of course, there’s the $60,000 in student loans that I’ve got to deal with.
Ideally, I’d like to stay where I am, and use my spare time to work on open source and entrepreneurial projects. I want to get the GBTC Estimator upgraded to a GBTC trader, and see if there’s any income to be generated there. I’d like to complete the Safe.Trade integration into CCXT. I’ve got the medical transportation company that I want to build a Django app for, and I’ve got another opportunity with a new friend who is very entrepreneurial. Other than that, I just plan on crawling the boards on AngelList and other local startup boards to see where I can join on as technical adviser.
Of course, all that goes out the door if I lose my job. I’m not sure how bad the situation is at work, since my boss doesn’t share anything other than “we need money”, and we haven’t brought on a new client in close to a year. We had a discussion about taking on software development work, but all I got was push back. He tells the team to “go out and sell”, and we’re all like “mhmmm”, but that’s all it amounts to. I’d just rather he furlough us all at this point.
I’d rather not turn this post into an obit for the company, but it’s been a zombie for some time. It’s like we’ve got just enough clientele to keep things from sinking completely, but not enough to grow. Which means of course, that it’s going to die, probably as soon as I leave. I told my boss that I wouldn’t abandon him after graduation, but I’ve been trying to lead and direct the company to where we need to be, and have been ignored too many times.
We currently have a client in the service industry which relies a lot on manual paper processes and third party vendors to manage their work order and invoices. I consulted them nine months or so ago about migrating their workflows to Microsoft Forms, Flow, and SharePoint Online. There was a lot of excitement and head-nodding, but nothing has come from it. Instead, one of their employees has been learning Django and building a pricing calculator. I got mad respect for them, and have been shooting the breeze with him about, making recommendations and the like. Now, however, it’s getting to the point where they’re asking questions about how to deploy this app, and I’m at a limit as to what I can do in a non-professional capacity. We decided to table discussions till next week.
One of the problems that I’m running into is around making this phase transition from one career to another. The crux of the problem is related to the difference between understanding something from a theoretical standpoint to actually having done that thing. Past performance, if you will. I ran into a concrete example of this the other day. We, Zombie, Inc., that is, had an opportunity with a prospect that needed a website update. They were using WordPress, and we identified a potential vulnerability via a web scan. The site template was very rough, and needed to be brought up to a more current aesthetic. The problem was that while I have plenty of experience managing WP sites, I have no portfolio of sites that I’ve built. And Zombie has zero performance that they can point to. So of course, nothing has come from it.
It seems the cure for this problem is just to do stuff for free, and then try to recoup payment for it on the back end. I think Tim Ferriss has an example from his life, back in the 90’s, where he would find businesses without an online presence, build pages for them and then approach them afterward to try and sell it. There’s similar examples, but they all depend on having the time to do the work up front.
That’s basically where I find myself right now. The “clients” that I have right now are little more than experiments to see if I can make a decent side hustle doing site management and consulting work. Monday, after I have put the final nail in my undergraduate degree, I will contact Zombie’s client and craft some sort of consulting deal that will benefit all three of us.
So we’ve begun week six. Writing has proved difficult recently, as I’ve been getting up roughly the same time as the girls and have been unable to focus on writing until later in the day, after my day seems to have filled up with tasks. Saturday marked the first real bit of restlessness I’ve felt since we started the lockdown, a bit of ennui and listlessness about what to do.
We’ve rearranged the room over the garage. My wife’s desk is setup and she’s able to telework. I took one of my old workstations and set it up for Elder. I tried using Wine for the first time, but had trouble with some fonts and wound up wiping it an installing Windows 10. I’m hoping she’ll take interest in computer art or music production, but she’s mainly interested in playing Roblox. I gave her a free pass yesterday and asked her what she wanted to learn about. She said “music”, so I threw on a YouTube video lesson for children.
She’s been accepted by the gifted program and will be going to the city’s gifted center for third grade. Her teacher called me Sunday to ask if I would be interested in letting her be part of a small group in the class that would be doing more advanced math, and of course I said yes. I’ve managed to get her to do piano without too much fuss, but I haven’t pushed too much. I can’t say for sure, but it seems that there’s been fewer tantrums.
We’ve discovered Amazon Music and that it has Trolls and Disney music, so the girls have been playing that a lot.
I’m in the midst of my final exam for my numerical methods class, and have been getting my solvers working. Right now the Gaussian elimination is the only one working, and I’ve got 3 more days to get the others working. The professor wants us to generate surface plots in Excel, of all things, and to turn those in for our answers. Since all the solvers are supposed to return the same results, I could just turn in the answers I’ve generated thus far, but I still need to turn the solvers in for assignment credit. The problem here is that I’ve built a large build and test suite in CLion, and my professor just wants a single CPP file that he can run in CodeBlocks. I’ve painted myself into a corner, but I’m not concerned with grades since I think the professor is going to grade on a massive curve.
One of the graduation requirements is financial aid counseling, and I got the first look at my student aid totals in a long time: over fifty-seven thousand dollars. There seems to be some discrepancies that I’ll need to review, but this is obviously a lot more than I was expecting. I hurt myself by taking cash payouts for personal expenses. These went to pay credit cards, and quite a bit to bitcoin. I’ve already accrued five grand in interest charges. It puts my post-graduation plans in a bit more context. The status quo will not hold.
I’ve got until next year before I’m expected to start paying these loans back, but the interest is well over four percent, so the first thing I’ll be looking to do is refinance.
I’ve decided that I need a proper professional presence online, so I’ve registered a few domains and started setting up a new CV site. This blog will remain separate for now, but I’ve started reposting some articles on Medium, and will be linking to my Github repo on it. I’ll worry about the ramifications of a recruiter seeing my Tweets and blog posts later. For now, the only thing that comes up when you Google my real name is my political work, so I’ve got to work on changing that.
I’ve also started trying to use LinkedIn more. There are a lot of jobs for software developers and engineers lately. About twenty new ones a day. I’m not saying I can take my pick, but there’s been about one or two each time I look that I’d be interested in. Not that I would necessarily be qualified for, but once I get through my exam and independent study requirements, I’ll be finishing up my resume and applying to some. Not that I really have any desire to work for another firm full time, but I doubt I would turn down an eighty thousand dollar a year position right now.
So the search has officially begun. I spent some time on AngelList, looking for opportunities, and sent a few messages out some founders. All equity stakes, unfortunately, but hopefully I’ll start the conversation rolling.
Also reached out to the team behind a new AI startup. It seems that they’re running various AI projects behind the scenes and offering an API for devs to get the results. Seems like an interesting platform play, providing AI as a service. Reminds me of what Twilio did.
Following the advice of Peter McCormick in The 10% Entrepreneur, I’ve started writing a professional biography. I uploaded my draft to AngelList, and will do so on LinkedIn as well. I have to be careful how I put things, cause I don’t want any blow-back from my boss right now. I don’t think it’s likely, given that they can’t survive without me, but I can’t afford to take chances. Anyways, the point of the biography is to build a coherent picture that brings together where I’m at now, where I hope to be, and encompasses my professional, academic, open source, and political contributions. I think I did a good job.
Today I will focus on making progress on client projects, and following up on any opportunities. One of my clients hasn’t paid in months, so it’s time to have a tough talk with them about the future. I have another advisory project I haven’t spoken to recently, and another relationship that I’m hoping to make the cornerstone of new business.
After assessing that, I need to make a few cuts, finance wise. I’ve been carrying Basecamp, Harvest, and an Adobe Creative Cloud subscription for some time, not to mention my Namecheap reseller account and AWS instance running the IDEX node. Unless I can secure some immediate funding source, I’ll have to cut. I think Basecamp may be willing to offer services for people affected by Coronavirus, but I doubt Adobe would do the same.
On a personal node, I should probably do the same with my Waking Up subscription. Sam Harris is gracious enough to allow people to request free subscriptions, so I need to do that. Taxes are also due in a month, so I need to start working on that. I’ll defer that till this weekend. I manged to eek out a refund last year because of business expenses, but I’m not sure I’ll be able to get away with that this year. We’ll see.
I’ve been reading The 10% Entrepreneur, and The Future is Faster Than You Think in bed the past few nights, hoping that it’ll prime my brain to come up with ideas for me to take in the next step in my career. I’m not going to share the crazy dreams I had last night, but I will share this idea, mostly for me to mull on and come back to in a few months.
This idea is for a IT support communications app utilizing voice and chat, with a little bit of AI thrown in for speech recognition.
The problem: I work for an MSP, and over time a number of customers have gotten ahold of my personal cell phone. This is bad. We have several endpoints, if you will, for clients to contact us: an official office number, which is routed to an answering service, a support email, which goes to our ticketing system; and a support desk number, which goes to our helpdesk partner. We have no texting capability.
The catalyst: During a client network outage, I called their ISP, and the hold message said that I could text an agent at a certain shortcode. I quickly hung up and texted the number, then proceeded to deal with the issue, asynchronously. I thought that it would be a powerful tool for us to use if we had similar agent capabilites.
I’ve done a bit of tinkering with Twilio’s platform, having used it experimentally during a few political campaigns. Their Flex platform is geared toward call centers and support agents, and all of their services have APIs, allowing it to be connected to other systems. (This interview with their CEO is enlightening.)
Solution: Build out a phone/text response tree and use it to replace our answering service with speech to text message relay, and also provide text messaging capabilities for clients as well. There are a number of ways to integrate this with our legacy ticketing system API for ticket creation, or, for ticketing creation and status changes in the other direction.
There’s another opportunity here as well. The firm I work for is a franchise, and there is so much redundancy built into the business model. Every independent franchisee has their own instance of our ticketing system, and has to hire or train their own resources to work with all of our different vendors: our RMM, PSA partners, cloud services (e.g. O365 various services), disaster recovery/business continuity, and so on. I don’t even want to try and count the number of vendors that I have to deal with. Personally, I’m well suited to this type of jack-of-all-trades position, but I’m at the point now where I brisk at having to learn or deal with some new system that doesn’t have APIs or programmatical interfaces. And through my interactions with other franchisees and techs in our Slack, I can tell that some of them are less than capable of handling some of these projects.
The onboarding process for our location was very difficult. We were pretty much handed some tools and left to figure them out for ourselves. There’s been some improvements in how this is handled more recently, but one of our vendor onboarding documents was near fifty pages of step-by-step instructions and screenshots.
I’ve tried to set up some automations internally, and tried to get traction among the other franchisees, but the appetite just doesn’t seem to be there. I just don’t think the owner community is really thinking along the same lines as me, and this is one of the main reasons why I don’t think the firm is a good fit for me any longer. I approached one of the home office leaders about using some of the API work I’m doing to do some cross-franchise data mining, and got dismissed out of hand.
I think there’s a huge opportunity to consolidate some of these roles and operations across the franchise system. In fact, I think that it’s the only way that some of the smaller franchises are going to survive. That said, I think the way the business model works, and the way the franchise system has been sold to the franchisees will allow these improvements to be made.
The system I’ve described above should allow multiple franchise locations to share the same dispatching and messaging contacts, and allow messages to be routed to the proper client owner. I will share this idea within the global group, to see if anyone is interested funding development of such a system.
The company that I work at is coming up on seven years old this winter. We’re a small managed service provider with about 4 employees and 25 or so clients. We provide IT support and project implementation services for small professional and service companies. We’ve been stagnant, growth wise, for the past three years or so, and my main focus in addition to taking care of our clients is refining our business processes so that we can scale to the next level. What we’ve been doing has brought us success, but it’s not enough to get us to where we want to be.
We’re part of a franchise system of independent operators all over the U.S. The home office is supposed to provide us with best practices and partner relationships, and the franchisees pool their purchasing power to get best deals with the partners. That’s how it’s supposed to work, anyways. What’s happened in practice is that the home office basically provides new franchise owners with a vendor for this, a vendor for that, and so on, and basically leaves the franchisees to themselves to figure out how to implement it. It’s completely inefficient. I can’t even begin to tell you how much time we’ve spent managing our RMM and PSA tools, or how much of my day to day is refining these various systems (some of which don’t have any API for automation control) to talk to each other.
Instead of pooling human resources, say to have a team of engineers that specialize in setting up firewall systems, for example, each location pretty much has their own teams. We rely on outside NOC and helpdesk partners to deal with first-line issues, and the local teams are supposed to be escalation support. But providing information to these various entities can be very difficult (ITGlue has helped tremendously!) but having a remote helpdesk is very frustrating for customers who expect some sort of continuity.
Unfortunately we’re just not able to provide that level of service for what clients are willing to pay. Especially the smaller clients. MSPs use a per-month contract billing, with rates for servers, workstations, and other IT resources, but that usually just covers keeping things running, remotely, and on site and project work is billed separately.
Things can really add up for clients, especially when they don’t follow our recommendations and shit goes south. Most of them are trying to balance the cost of having their own in-house IT resource, but hardware, software and human resource costs can quickly add up. This is even more true when you consider regulatory and compliance requirements. It’s really hard.
And companies that skimp on these costs always pay for it. Always. I’ve had my fair share of ransomware breaches, but one that I saw this week really took the cake. An firm who we have done business with in the past, that we’ve been under a limited engagement with, had a really bad attack which took down their entire Windows domain: three servers, including AD, Exchange, SQL, file services, and a custom database application. We stopped doing business with them three years ago because it was always a challenge to justify what needed doing over there, and things were usually such a matter of urgency that we would be forced to do things to keep them running. And then we would have to spent weeks having to pull teeth to get paid. We finally said enough is enough and just walked away.
So we got a call from them a few weeks ago. Turns out they had pissed off another MSP, and needed help. They had been through several in-house IT resources, but they needed RMM monitoring, AV and patch management stuff that we would provide. But because they were in dispute with the old IT company, we weren’t able to get access to their backup and data continuity appliance.
Long story short, they got hit earlier this week and didn’t have backups for half their shit. I had convinced their in-house person that they really needed to get some sort of local backup, and thankfully they followed my advice. But it was really too little, and they’ve spent the last 72 hours trying to recover. And let me tell you, it was the most stress-free disaster recovery that I’ve ever dealt with. I’ve damn near had panic attacks and probably lost years off my life from the stress of dealing with my own share of these disasters. Sometimes they were self-inflicted, other times not. But since I wasn’t the one holding the bag, I was chill as fuck.
I’ve saw the writing on the wall for MSPs some time ago. I don’t know if it will be ten years or when, but the business model is going to approach a race to the bottom. And our local market is already saturated with 4 or 5 decent competitors, and many more not so decent. Internal conversations around the future of our firm talked a lot about compliance auditing for DOD/NIST, and the question we’re struggling with now is whether we want to be an MSP that does compliance, or a compliance firm that does MSP. My gut tells me to go where others aren’t. Which is why I’m focusing my time on process automation, combining applications via API.
I was able to list several things to our no list, things we’ve done in the past that have gotten us into trouble in the past. That means setting boundaries for business that we deal with, and will likely involve cutting some of our clients who aren’t growing with us or don’t see the value of the service we provide. It means converting our services to product offerings in order to differentiate ourselves from the competition. And it means automating our processing so we’re not making the same decision over and over again.
Tomorrow marks the start of my last year at university, where I’ll be finishing up my bachelors degree in computer science with a computer science minor. I’m only attending half-time, and the two of the four classes I need to finish are a professional workforce development course. Obviously, this is going to take a good deal of time away from everything else that I’ve been doing, so I’ve labored to unload as many projects that I can. That said, these are writing intensive courses, and I don’t know what kind of time commitment that’s going to take. Obviously, taking thirty to sixty minutes a day is going to be hard to fit in, but I’m going to be staying on top of the assignments to be able to fit that in.
That said, there may be room for crossposting. In the past, I’ve published writing assignments from class to Facebook or Medium in the past, so I expect I’ll find ways to kill two birds with one stone. That said, one of the first tasks is to share my thoughts on what it means to be a professional. Specifically, the characteristics a true professional must have.
Integrity is doing the right thing even when no one is watching.
C.S. Lewis
My dad taught me his work ethic, and while I’ve been slow to get going some times, I’ve I’ve never had a problem focusing on a task once I’d made my mind up to execute. Obviously, there’s a difference between personal tasks and professional ones, but I’ve always hustled my butt off. Always. Even when I didn’t have the ability, or wasn’t the best, I could still keep going, driving toward the finish line. But beyond the drive, integrity is probably the most important trait one can have. Your reputation takes a lifetime to build, but can be destroyed in an instant. And taking shortcuts, or otherwise cheating a client or task will come back to haunt you.
There are lots of other answers that people will give as an answer to this question, but I think the question is the wrong one. When people talk about characteristics, they’re really discussing a trait, or a skill. One of the most valuable lessons that I’ve learned lately is about choosing the people that I work with. Whether you’re hiring for a position, taking on a client, or choosing a new job, the most important questions that ultimately need to be asked are around values.
Values are the deep-seated beliefs that motivate behaviors; people will fight for their values, and values determine people’s compatibility with others. Abilities are ways of thinking and behaving. Some people are great learners and fast processors; others possess common sense; still others think creatively or logically or with supreme organization, etc. Skills are learned tools, such as being able to speak a foreign language or write computer code. While values and abilities are unlikely to change much, most skills can be acquired in a limited amount of time (e.g., most master’s degrees can be acquired in two years) and often change in worth (e.g., today’s best programming language can be obsolete in a few years). It is important for you to know what mix of qualities is important to fit each role and, more broadly, with whom you can have successful relationships. In picking people for long-term relationships, values are most important, abilities come next, and skills are the least important.”
Ray Dalio – Principles, #45
I’ve been at my current firm for almost seven years now, and I’ve sat on the side through a number of hiring interviews during that time. Ultimately we’ve been disappointed with those hires that we’ve taken on, and I couldn’t really understand why until I read Dalio’s principles a few months ago. Every time I sat at that table with someone’s resume in hand, I was always focused on the skills. We were hiring for a position, an immediate need. And while I may have touched briefly on some of those deeper abilities, we almost never discussed the values that drove a person. A lot of your standard interview trick questions may have been originally designed to get into some of those values, but I think they lost meaning the more they became rote. And it’s hard to get to know someone in that short timeframe.
So while we may have chosen hires that were capable of performing the skills that were needed at the time, we handicapped our future growth. We wound up with employees who weren’t motivated to keep learning new skills as business needs changed, that were using the workplace as a dating pool, or who were incapable of documenting their work properly. And make no mistake, I’m no angel myself. Most of the jobs I’ve had over the years have been failures. And this may be my privilege talking, but I’m not afraid to be fired any more. And I’m not afraid to fire a client if they don’t align with our values. I’m at the point now where I can say ‘no’. I’ve realized that a lot of what comes my way is going to distract me from what really matters, and what I’d rather be working on.
I’m forty years old and still trying to figure out what my personal mission statement is. I may not be able to spell it out, but it’s there. I think ultimately it’s about service, and passing on what one has learned to others and helping them along. It’s about building connections and community. Hoarding knowledge is ultimately futile. I think lately I’ve been thinking that if I have an idea and someone else can do it better, then by all means, let them. I’ve got to focus on the things that I can do better than anyone else. What’s my niche? If someone brings something to me, the first thing I ask is ‘am I the only one that can do this,’ and that usually determines my answer. There’s other factors to be considered, of course, but I try to stick to that as much as possible these days.
One last concept that I’ll leave here is the concept of life as a multi-armed bandit problem, where we’re always exploring and experimenting and figuring out ways to exploit that knowledge that we’ve gained. Having this framework in mind and knowing when it’s time to put in the work to experiment build those relationships and reputation, and when it’s time to focus on that one thing that is going to bring you success — that’s key.
But hey, I’m no expert yet. I’m still learning too.
