A big part of the world of programming is understanding concepts. Learning to learn is something every programmer has done. How do you learn new concepts? What are your methods for improving efficiency and elevating your craft? You would think these would be the biggest questions in the hiring process. However sadly they aren’t.

This concept can be proven by seeing how a lot of programming interviews go. Asking a programmer to interview in the most alien way possible. No intellisense, no Google foo, no man pages. The worst interview tech question I ever had, “How would I learn to use a tool without the man page or Internet?” I don’t know, how would I compile a program without a compiler. We can all do great things with constraints but by taking away the tools that make doing what we do efficient what are we really learning about a candidate.

My last gig I was running a small team of Python programmers. I was in charge of interviewing and acquiring new talent for the tech side of our business. What were the things I interviewed for? I mostly looked for how the candidate worked and perceptually understood things. Problems would be presented to them as an expected input and expected output and a broken algorithm that needed to be repaired. Most of the company’s time was spent visiting problems like these. The algorithm had no unit tests. Did the candidate ask about unit tests? Did the candidate poke at the code to brute-force their way to a solution? A surprisingly high amount did. The process was unconstrained, they could search the internet all they wanted. The reason why? Because they wouldn’t find some quick Fizz Buzz solution by doing so.

The broken algorithm I wrote for the interview was a small one, all it did was look up a phone number by name and the data was just stored in a dictionary. The code was just a little 6 line Javascript file. No frameworks were utilized. If the candidate knew some how to conceptualize what the code was trying to do they should have no issue debugging and fixing it. In the time the candidate spent debugging that code, I learned everything I needed to know about the them. Even if they had never wrote a line of Javascript, did the candidate look up JS and learn what they needed? Frankly JS syntax was the least important aspect of the interview to me. Runtime errors can help you resolve many of those issues. The time spent working on the code was also not that important. However, if they spent 15 minutes and not really talking it out, not really searching for anything or explaining where they felt the problem was. Then they were probably not that great of a candidate. While you might know answers to interview questions by heart, if you can’t produce that skill in the real world, you aren’t likely to be successful.

Sadly, the interview world for tech isn’t getting any of this. Qualifications are based on knowing specific syntax or languages that is the company’s current flavor of the month. What happens when that language is out of vogue or a new concept comes around that is better then what they currently use? Will they keep requiring their tech talent to hold this outdated syntactical knowledge or will they hope that their talent would re-conceptualize the problem in the new language or new toolset?

Hire for concepts not for syntax.

I’ve been generating email on my own services for quite some time. 20 years ago sending email was easy. I could fire up sendmail and do something like:

sendmail -f [email protected] [email protected] <<EOF
Subject: Hello my friend
From: Steve Jobs <[email protected]>

I love you
EOF

Easy as pie. The problem with this ease is I could easily set that -f argument to any email address I like. On the other end this email would look legitimate so long as I didn’t abuse the receiving email provider with my IP. We know where this story goes. Spammer pretends to be CEO, asks finance to write a check to ABC company and finance made the blunder of actually trusting what they saw in an incoming email.

The 2000s are filled with poorly thought out solutions to these kinds of problems. Most of them either revolved around scanning messages for certain characteristics, phrases or expressions and then there were the IP reputation services. PGP as an attempt to secure message authorship, but required active participation from all receivers. While these ideas met limited success there were many casualties along the way. There are also the “double knock” type email systems where your intended party’s service will send an automated message back to you and ask you to verify you are human before your message will be released from a queue. Solutions like these cause all sorts of unintended side effects, especially if the email can never be seen by the intended party by perusing their spam folder.

Enter Sender Policy Framework, or SPF, is a way to use something authoritative like DNS to tell mail providers where they can expect mail to come from for a domain. An email from a domain without SPF available might seem sketchy to some email providers. Add an SPF policy and you get a gold star from the likes of Google, Outlook and other free e-mail providers.

v=spf1 ip4:127.0.0.1 ip4:127.0.0.2 include:mx.example.com include:mx2.example.com -all

What does this mean? v=spf1 is the version of SPF we are utilizing. As of 2021 this will always be v=spf1. Next in importance is the end of this line. -all tells other providers to FAIL any email that doesn’t conform to this policy. If you domain does not serve email at bare minimum you should set this record to:

v=spf1 -all

The above basically says, reject all email claiming to be from my domain. Optional elements of the SPF record are “a” “ip4” “ip6” “mx” “ptr” “exists” and “include”.

• A – Tells providers that mail coming from the same source as the main A record for a domain should be accepted.
• IP4 (followed by a dotted IP4 address) tells providers what IPs or Networks can send email under our domain
• IP6 – is the IPv6 equivalent of the above.
• MX – tells providers that mail coming from servers in our MX record are safe.
• PTR – tells the provider that rDNS for a client is enough to allow email (this should be avoided)
• EXISTS – uses SPF macros to match address
• INCLUDE – includes the referenced domain’s SPF policy.
• ALL – matches everything, this is usually why ALL is defined at the end of a record.

Next are the qualifiers:

• + : Tells other providers that the rule is passing, lack of a qualifier defaults to this qualifier
• – : Tells other providers to fail on this rule.
• ~: Soft fail, typically quarantine the message
• ?: Neutral, is typically interpreted like no policy even exists.

Now that we know all of this we can interpret this line:

v=spf1 ip4:127.0.0.1 ip4:127.0.0.2 include:mx.example.com include:mx2.example.com -all

Version 1 Pass IPs 127.0.0.1 and 127.0.0.2 and include the SPF records from the domains mx.example.com and mx2.example.com, Reject everything else.

Now that we know SPF, problem solved right? We can now all implement SPF and never look at this problem again. Well, no, we need to apply a cryptographic signature to emails because how do we know that the email that we sent hasn’t been altered to look legitimate? Once it leaves my outbox, I trust many people with my email and any one of them could make an alteration to the email. I am sure you are familiar with the concept of a Man in the Middle attack. This is what Domain Keys were meant to solve.

Back in the mid aughts Yahoo was in the constantly frustrating position of being a spam target as such they saw a growing problem of manipulated emails. They brought out their own standard to resolve these issues. The problem was that not many people were aware of Domain Keys and other providers like Cisco were also trying to solve the issue of manipulated emails. They called their system Internet Identified Mail. Instead of competing to solve this problem they opted to join forces and create *drum roll* Domain Keys Identified Mail. We all know this today as DKIM, but this would solve the issue of mail authorship identity.

DKIM solves this by issuing a public key that is also placed on the authoritative DNS of our domain. Are you detecting a theme here? We also need to cryptographically hash our email at the source. Many use a tool called opendkim which is a filter placed in a mail server like postfix. A private key is used to generate a hash that is added to the header of our message. The header is labeled DKIM Signature. Inside this signature we define which header fields we want hashed, the “From” field is the only required header field. Along with the cryptographic algorithm in use. From Wikipedia:

DKIM-Signature: v=1; a=rsa-sha256; d=example.net; s=brisbane;
     c=relaxed/simple; q=dns/txt; [email protected];
     t=1117574938; x=1118006938; l=200;
     h=from:to:subject:date:keywords:keywords;
     z=From:[email protected]|To:[email protected]|
             Subject:demo=20run|Date:July=205,=202005=203:44:08=20PM=20-0700;
     bh=MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTI=;
     b=dzdVyOfAKCdLXdJOc9G2q8LoXSlEniSbav+yuU4zGeeruD00lszZVoG4ZHRNiYzR

The important elements of the above are version (in this case 1) , algorithm (in this case rsa-sha256) domain (example.net) selector (The DNS record holding the public key) bh and b are our body header and body hashes. The mail provider will then use the public key along with elements of the message to hash and compare the result. This TXT DNS entry will look something like this:

Name: brisbane._domainkey.example.net.
Value: v=DKIM1; k=rsa; t=s; p=<public key>

If the hashes match, we have a winner and it passes DKIM, if they are mismatched then it fails. The absence of DKIM is considered, as of 2021, neutral because not everyone has caught on to the importance of DKIM.

However, a new problem is coming. We have these great tools to help fight fraudulent email but setting all these things up is confusing and every mail provider could weight certain aspects of these tools more than others. On top of this, once our email messages are delivered to another mail provider we really have no idea what happened to that message. Was it delivered? Did our broken DKIM implementation block it? What if Kevin in support decided to setup a newsletter service for his customers and neglected to tell IT that they need to add a new provider to the SPF record. He is happily toiling away building these newsletters that are never seen because they all go straight to spam or are outright blocked and IT is unaware because who would ever report this issue.

Enter Domain-based Message Authentication, Reporting and Conformance, boy that was a mouth full. DMARC was created to solve this issue. We setup a new, oh you guessed it, DNS record to tell email providers what to do with our broken email systems. On top of this we also define a reporting email to send helpful reports about the activity of email carrying our domain. This is an important point because any number of email systems could be claiming to be an SMTP provider of our domain. Some of these systems are legitimate some are not. Using a DMARC policy we can tell other email providers what to do with email that doesn’t pass DKIM or SPF. We also can get daily reports on what is happening with emails claiming to be from our domain.

v=DMARC1;p=quarantine;sp=reject;pct=100;rua=mailto:[email protected];

v Defines the version of DMARC we are using, p is the policy, sp is the subdomain policy pct is the amount of emails to apply the policy to and finally rua is the address to send reports to.

Generally you want to be fairly permissive with DMARC while you figure things out. I recommend utilizing the quarantine policy until you have things working as they should, then move to reject. You can then examine your DMARC reports and determine if things are working as they should. Once you are certain that the systems that should work do, move to a reject policy.

What about these DMARC reports, what do they say exactly? Well let’s look…

<policy_published>
  <domain>example.com</domain>
  <adkim>r</adkim>
  <aspf>r</aspf>
  <p>quarantine</p>
  <sp>quarantine</sp>
  <pct>100</pct>
</policy_published>

This block tells us what the mail provider detected as our current policy. This policy is defining a relaxed posture towards dkim and spf issues. Additionally when a problem is detected we are asking them to only quarantine messages which means they typically go to the spam folder.

adkim and aspf both default to a relaxed posture “r”. You can define a strict one by setting your DMARC option adkim=s;

Next you will find a record…

<record>
  <row>
    <source_ip>127.0.0.1</source_ip>
    <count>36</count>
    <policy_evaluated>
      <disposition>none</disposition>
      <dkim>pass</dkim>
      <spf>pass</spf>
    </policy_evaluated>
  </row>
  <identifiers>
    <header_from>example.com</header_from>
  </identifiers>
  <auth_results>
    <dkim>
      <domain>example.com</domain>
      <result>pass</result>
      <selector>mail</selector>
    </dkim>
    <spf>
      <domain>example.com</domain>
      <result>pass</result>
    </spf>
  </auth_results>
</record>

Seeing this record tells us that 127.0.0.1 is a good source for our domain and has successfully sent 36 emails to this specific provider. DKIM and SPF pass and all mail has made it to their intended parties. Note: that emails could still be filing as spam for not meeting other heuristics of the provider or being marked as spam by individuals. So what happens with problem sources?

<record>
  <row>
    <source_ip>192.168.1.1</source_ip>
    <count>5</count>
    <policy_evaluated>
      <disposition>quarantine</disposition>
      <dkim>fail</dkim>
      <spf>fail</spf>
    </policy_evaluated>
  </row>
  <identifiers>
    <header_from>example.com</header_from>
  </identifiers>
  <auth_results>
    <spf>
      <domain>example.com</domain>
      <result>softfail</result>
    </spf>
  </auth_results>
</record>

This tells us that a server located at 192.168.1.1 tried to send 5 messages as us to this provider. Since they were not in our SPF and likely could not have provided a valid DKIM hashed email they failed and according to our DMARC policy were properly quarantined.

We setup all three and now we can continue sending automated email and ride off into the sunset. Right? Well no, automated email can easily break DKIM. Even if you setup everything perfectly, if you do not know how to format a message to conform to SMTP limits a message can be altered in transit, breaking DKIM signatures. See, still broken. SMTP has limits and unfortunately a lot of tools have been placed over the top of this unruly protocol. First and foremost you must ensure that your systems are using the correct sender. Use -f to set the appropriate email address that contains our real domain, don’t just let the system set whatever it likes because it will just use the current user @ hostname as the author. Even if you set a “From” field in the body of your email, the “-f” flag is critically important in identifying who the real author should be. I know this sound counter-intuitive but “From” “-f” “Return-Path” “Reply-To” all do different things.

Additionally SMTP has line length limits. It’s easy for the body of a message to contain log lines or html content that could hit these limits. The general rule I follow is no line longer than 900 characters. As a general rule you can opt to Encode the message to protect it from these limits. Encode protocols that I quickly reach for are:

Content-Transfer-Encode: base64
# or
Content-Transfer-Encode: quoted-printable

Wait, why does this matter, you probably thought I was talking about troublesome concepts like DKIM or DMARC. It matters because if you fail to limit line length some SMTP server along the chain are going to force the change for you. When that enforcement occurs it will break the DKIM hash because now some other SMTP server is going to enforce a line length restriction which will alter the original message. In the old days this alteration would have gone unnoticed but with cryptographic integrity on the line, it will be noticed.

Base64 encoding is probably the easiest to understand, all we need to do is encode the material and split it.

cat file.log | base64 | fold -w 76

Now you just need to set a Content-Transfer-Encode header in the email along with the original Content-Type.

Another option is to use Quoted Printable, this is typically used for HTML emails so I won’t go into examples but I can tell you how it works. You will want to fold all lines to be no longer than 76 chars and you will need equal (=) encode lines that have been wrapped. Any = characters found in the source document will also need to be encoded to =3D

An example:

Content-Transfer-Encoding: quoted-printable

<html><head></head><body style=3D"font-family:Arial;"><table align=3D"cen=
ter" border=3D"0" cellspacing=3D"0" cellpadding=3D"0" width=3D"60%" bgcolor=
3D"#FFFFFF" style=3D"background-color:#FFFFFF;table-layout:fixed;-webkit-te=
xt-size-adjust:100%;mso-table-rspace:0pt;mso-tablelspace:0pt;-ms-text-size-=
adjust:100%;min-width:500px"><tr><td><table align=3D"center" border=3D"0" c=
ellspacing=3D"0" cellpadding=3D"0" width=3D"100%" bgcolor=3D"#EDF0F3" style
...

Basically = allows you to define any character but you must use this escaping policy for any non-printable ASCII characters. You can read more in the wiki article linked below.

Hopefully these hints will help someone who runs into these issues in the future. Some resources that will help you in your journey.

1. https://en.wikipedia.org/wiki/Sender_Policy_Framework
2. https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail
3. https://en.wikipedia.org/wiki/DMARC
4. https://en.wikipedia.org/wiki/Quoted-printable
5. https://en.wikipedia.org/wiki/Base64

So as I have been working on building and evolving several of my side projects into something that is somewhat more cohesive and organized.  I have started to think about what is next in online media or even just internet technology.  Since 1998 I have known the power of the internet as a tool to bring back influence to the consumer.  If you think about the way traditional media has worked, you will find that largely they were the single source of information.  We relied on Newspapers, Radio and Television stations to tell us what is going on in our world and we expected that it would be relevant.

As society has grown something unconsciously started occurring.  Since so much was happening everywhere it was difficult for traditional media outlets to, digest it first, then put it out in a meaningful way.  Organizations got larger to handle all of the information and through media ownership reforms we had consolidation in the environment.  This new environment has created a difficult situation.  Now suddenly what used to be news, is no longer that relevant.  It’s not about a traffic accident down the road from your house.  It is now more about how the president handles peace talks.  Or how a government halfway around the world is doing business in your own local government.

This has left a void in communities within large markets.  I am sure that peace talks are important to me, but are they as important as the fact that 5 of my neighbors down the road were burglarized?  How about the fact that a local shop owner is having a fire sale and it just so happens to be a store you visit all the time?  This information never makes it to you because it’s relevance by world standards is low.  Traditionally, media had to dedicate an employee to write up the story.  An editor/producer had to determine it’s relevance.  Then time was spent distributing this information.  At the end of the day, it was left up to the consumer of the program or newspaper to filter out what matters to them most.

The power of the Internet and computing allows us to let sophisticated algorithms do the work for us.  They can do this work based on our preferences.  If you have ever talked with anyone in advertising, they will tell you the most important thing in their business is understanding behavior.  This has been true of media for decades and true for business for centuries.  So what is happening now, is a transition to allowing the internet to track and determine preference based on behavior.  Your cell phone is the first step to allowing this.  In a few years time, I imagine that the next big thing is a basic tracking setup on your phone.  Some might argue that this already occurs, but I imagine that this tracking will be hidden under the guise of something cool or trendy for young people.

Companies are already hard at work trying to feel around how invasive they can be.  You have several apps that the user can intentionally allow location based information to be sent.  Some are disguised as games and others are marketed as beneficial because it allows your friends to know when you are nearby or where you are located.  This is why privacy is becoming more and more of a concern but that is another article.

The next evolution won’t require as much investment in time as I see it.  The next evolution has already begun.  Web 3.0 as some have called it.  You might find some overly technical explanation for it on the web but basically it will be about the silent web presence.  How the web is going to be a part of the physical world and how it will interact with you in that world.  10 years ago it was talking refrigerators, however, I think that is a little overboard.  I imagine a world that reacts and knows you and mirrors you.  A dating experience involving going to a local hangout and having a device tell you that a potential mate is there as well.  A social experience where behaviors are shaped by the way your preferences cross with other individuals.

Your smart phone won’t only know that you need something to do tonight, but it will know where you should go to have the highest potential for meeting new people because, based on past behaviors, it calculates that you might need new friends.  It might remind you to lock your door because not only did you forget, but there were 5 burglaries down the street from you.

These ideas are both scary and compelling.  However, it has already happened with intangible properties like music.  Could you see yourself going back to the old model of actually having to go to a store to not only find a good artist to listen to, but to also purchase your music?  For most, I think not.

So yes, textclad went away for a while.  Not certain that it is back for good, but it has a new home.  I have effectively set up a cloud solution and finally after almost 2 years I am at a point where it really makes sense to self colo.  75% of the way there.  I am also now responsible for the direction of a new company so we shall see how that all turns out.  My first goal was to slash costs and I think I have done that.  I am also steering things towards a more global market.

The original focus of the company was on game tools.  However that is a tough market and many of the tools fall in to open source.  So direction and winds are driving me to look for new markets to penetrate.

So using a pretty fast internet fiber connection has it’s benefits and I am investing in some of that.  Some of my projects will be getting some much needed overhauls as I bring much of the backend in house.  I have amassed the capability to store up to 5TB of data in a closet utilizing a RAID array and SAN system as well as a barebones system with really impressive hardware all for the benefit of building virtual machines and shutting down my reliance on some external systems and providing some room for growth.  Wish me well.  😉

I found it interesting today that while I was setting up some shares on my Mac Book Pro I found an interesting icon for my windows machine in the mac OS.  Apple seems to have a grim view of MS even though it hasn’t used the type of BSOD screen used in the icon since Windows ME.  Makes me wonder if Apple Shares or other Apple devices like the iPod will or should have the sad looking mac man as their icon on your Windows desktop.