justin․searls․co

How can I ensure e-mail delivery?

Prompt:

A customer of betterwithbecky.com has reported that they aren't receiving authentication e-mails, and therefore cannot access their account. The customer's e-mail address uses a custom domain but is hosted by Google. The app sends e-mail from app-mail.betterwithbecky.com via Amazon SES, which reports everything is configured correctly.

Attached are the complete DNS Zone records for betterwithbecky.com as well as the headers of a raw e-mail message I just received after attempting to log in via Gmail.

Here's what the computer said:
(Generated by OpenAI o3-mini-high)

Analysis of Email Deliverability Issues for betterwithbecky.com

Background and Symptoms

The domain betterwithbecky.com uses Amazon SES to send transactional emails (login verification messages) from support@betterwithbecky.com. Amazon SES is configured with a custom MAIL FROM subdomain (app-mail.betterwithbecky.com), and SPF, DKIM, and DMARC DNS records have been published. Amazon SES reports these emails as “delivered” (the recipient’s mail server accepted the message with a 250 OK), yet multiple users – especially those with Gmail addresses – do not receive the emails in their inbox or spam folder. This suggests that Gmail is likely accepting the messages but then filtering or discarding them internally. We need to examine possible causes, including authentication alignment, Gmail’s filtering criteria, and any configuration or content issues that could lead to silent dropping of emails.

Email Authentication Configuration (SPF, DKIM, DMARC)

Proper email authentication is the foundation of deliverability. In this case, betterwithbecky.com has implemented all three mechanisms:

Let's dive in and find out…

The best argument against e-mail based authentication is that it’s impossible to guarantee the delivery of e-mails. Once a user says they've checked spam, there's basically nothing else you can do for them beyond checking your configuration for the millionth time.

I don't mean to pick on Pawel Brodzinski in this blog post, but I stopped reading right at the top:

In its original meaning, Kanban represented a visual signal. The thing that communicated, well, something. It might have been a need, option, availability, capacity, request, etc.

I hate to come off as a pedant here, but something that's always annoyed me about the entire family of Lean practices in the Western world is the community's penchant for its uncritical adoption of regular-ass nouns and verbs from Japanese. Lean consultants have spent literal decades assigning highly-specific nuanced meanings to random words, and if you actually listen to anyone introducing Lean, it's hard to go 5 minutes without getting the icky sense the use of those words is being deployed to trade on appeals to nonsensical Oriental exoticism. I've lost track of how many times I've heard something like, "according to the ancient Japanese art of Kaizen," or similar bullshit.

It's true that Lean's existence is owed to the work of luminaries like Deming, Ohno, and Toyoda and their development of the Toyota Production System, but what eventually grew into the sprawling umbrella term "Lean" was based on surprisingly brief and incomplete glimpses of those innovations. As a result, the connective tissue between Lean as it's marketed in the West and anything that ever actually happened in Japan is even more tenuous than most Lean fans probably realize. So the fact that everyone carries on using mundane Japanese words as industry jargon makes even less sense.

For example, here are some words Lean people use and what they actually mean:

And so on.

As an entitled white man, I'll be the first to admit I don't lose much sleep over cultural appropriation. I'm just saying, if you're trying to come up with a name for a specific concept or process, remember that existing words have meaning before cherry-picking a noun from a foreign language textbook and calling it a day.

UPDATE: Just as I was worried I might have been a bit too harsh here, I realized his blog has comments.

This one is just incredible:

A post it note is not a kanban

Theo, you might have to reconsider your idea of “idiocy”, potentially in front of a mirror. “Kanban” is not a noun so of course a post-it can’t be one. The concept originated from Japan (Toyota factories to be specific) so it makes absolute sense to use the original word. Their method did not use a signboard at all, Kanban is the system, which you would learn with a couple minutes of focused googling.

Of course, open a dictionary and you'll see that kanban (看板) is categorized under meishi (名詞), which (unless the Lean folk have some other made up definition for it), means noun.

Well, Theo, we use a Japanese name because that’s where it came from. Have you ever heard of a tsunami, or kamikaze, or sushi? These are also Japanese words we use in the English which have more nuanced meanings than just googling their “literal translation”.

Additionally, I can understand that being as unintelligent as you are must be difficult but if you try your hardest you might be able to google “kanban” and “signboard” to learn that one refers to a methodology and the other does not.

For example, real expert Lean practitioners know that "ahou" (阿呆) refers to observing a mistake repeatedly and forming an expensive twelve step correction plan, even though its literal translation is "idiot."

Ruby makes advanced CLI options easy

If you're not a "UNIX person", the thought of writing a command line application can be scary and off-putting. People find the command line so intimidating that Ruby—which was initially populated by a swarm of Windows-to-Mac migrants—now boasts a crowded field of gems that purport to make CLI development easier, often at the cost of fine-grained control over basic process management, limited option parsing, and increased dependency risk (many Rails upgrades have been stalled by a project's inclusion of another CLI gem that was built with an old version of thor).

Good news, you probably don't need any of those gems!

See, every UNIX-flavored language ships with its own built-in tools for building command line interfaces (after all, ruby itself is a CLI!), but I just wanted to take a minute to draw your attention to how capable Ruby's built-in OptionParser is in its own right. So long as you're writing a straightforward, conventional command line tool, it's hard to imagine needing much else.

This morning, I was working on me and Aaron's tldr test runner, and I was working on making the timeout configurable. What I wanted was a single flag that could do three things: (1) enable the timeout, (2) disable the timeout, and (3) set the timeout to some number of seconds. At first, I started implementing this as three separate options, but then I remembered that OptionParser is surprisingly adept at reading the arcane string format you might have seen in a CLI's man page or help output and "do the right thing" for you.

Here's a script to demo what I'm talking about:

#!/usr/bin/env ruby

require "optparse"

config = {}
OptionParser.new do |opts|
  opts.banner = "Usage: timer_outer [options]"

  opts.on "-t", "--[no-]timeout [TIMEOUT]", Numeric, "Timeout (in seconds) before timer aborts the run (Default: 1.8)" do |timeout|
    config[:timeout] = if timeout.is_a?(Numeric)
      # --timeout 42.3 / -t 42.3
      timeout
    elsif timeout.nil?
      # --timeout / -t
      1.8
    elsif timeout == false
      # --no-timeout / --no-t
      -1
    end
  end
end.parse!(ARGV)

puts "Timeout: #{config[:timeout].inspect}"

And here's what you get when you run the script from the command line. What you'll find is that this single option packs in SEVEN permutations of flags users can specify (including not setting providing the option at all):

$ ruby timer_outer.rb --timeout 5.5
Timeout: 5.5
$ ruby timer_outer.rb --timeout
Timeout: 1.8
$ ruby timer_outer.rb --no-timeout
Timeout: -1
$ ruby timer_outer.rb -t 2
Timeout: 2
$ ruby timer_outer.rb -t
Timeout: 1.8
$ ruby timer_outer.rb --no-t
Timeout: -1
$ ruby timer_outer.rb
Timeout: nil

Moreover, using OptionParser will define a --help (and -h) option for you:

$ ruby timer_outer.rb --help
Usage: timer_outer [options]
  -t, --[no-]timeout [TIMEOUT] Timeout (in seconds) before timer aborts the run (Default: 1.8)

What's going on here

The script above is hopefully readable, but this line is so dense it may be hard for you to (ugh) parse:

opts.on "-t", "--[no-]timeout [TIMEOUT]", Numeric, "Timeout …" do |timeout|

Let's break down each argument passed to OptionParser#on above:

  • "-t" is the short option name, which includes a single a single hyphen and a single character
  • "--[no-]timeout [TIMEOUT]" does four things at once:
    • Specifies --timeout as the long option name, indicated by two hyphens and at least two characters
    • Adds an optional negation with --[no-]timeout, which, when passed by the user, will pass a value of false to the block
    • The dummy word TIMEOUT signals that users can pass a value after the flag (conventionally upper-cased to visually distinguish them from option names)
    • Indicates the TIMEOUT value is optional by wrapping it in brackets as [TIMEOUT]
  • Numeric is an ancestor of Float and Integer and indicates that OptionParser should cast the user's input from a string to a number for you
  • "Timeout…" is the description that will be printed by --help
  • do |timeout| is a block that will be invoked every time this option is detected by the parser. The block's timeout argument will either be set to a numeric option value, nil when no option value is provided, or false when the flag is negated

If a user specifies multiple timeout flags above, OptionParser will parse it and invoke your block each time. That means in a conventional implementation like the one above, "last in wins":

$ ruby timer_outer.rb --timeout 3 --no-timeout
Timeout: -1
$ ruby timer_outer.rb --no-timeout --timeout 3
Timeout: 3

And this is just scratching the surface! There are an incredible number of CLI features available in modern Ruby, but the above has covered the vast majority of my use case for CLIs like standard and tldr.

For more on using OptionParser, check out the official docs and tutorial.

Programming is about mental stack management

The performance of large language models is, in part, constrained by the maximum size "context window" they support. In the early days, if you had a long-running chat with ChatGPT, you'd eventually exceed its context window and it would "forget" details from earlier in the conversation. Additionally, the quality of an LLM's responses will decrease if you fill that context window with anything but the most relevant information. If you've ever had to repeat or rephrase yourself in a series of replies to clarify what you want from ChatGPT, it will eventually be so anchored by the irrelevant girth of the preceding conversation that its "cognitive ability" will fall off a cliff and you'll never get the answer you're looking for. (This is why if you can't get the answer you want in one, you're better off editing the original message as opposed to replying.)

Fun fact: humans are basically the same. Harder problems demand higher mental capacity. When you can't hold everything in your head at once, you can't consider every what-if and, in turn, won't be able to preempt would-be issues. Multi-faceted tasks also require clear focus to doggedly pursue the problem that needs to be solved, as distractions will deplete one's cognitive ability.

These two traits—high mental capacity and clear focus—are essential to solving all but the most trivial computer programming tasks. The ability to maintain a large "mental stack" and the discipline to move seamlessly up and down that stack without succumbing to distraction are hallmarks of many great programmers.

Why is this? Because accomplishing even pedestrian tasks with computers can take us on lengthy, circuitous journeys rife with roadblocks and nonsensical solutions. I found myself down an unusually deep rabbit hole this week—so much so that I actually took the time to explain to Becky what yak shaving is, as well as the concept of pushing dependent concepts onto a stack, only to pop them off later.

The conversation was a good reminder that, despite being fundamental to everyday programming, almost nobody talks about this phenomenon—neither the value of mastering it as a skill or its potential impact on one's stress level and mood. (Perhaps that's because, once you've had this insight, there's just not much reason to discuss it in depth, apart from referencing it with shorthand like "yak shaving".) So, I figured it might be worth taking the time to write about this "mental stack management", using this week's frustrations as an example with which to illustrate what I'm talking about. If you're not a programmer and you're curious what it's like in practice, you might find this illuminating. If you are a programmer, perhaps you'll find some commiseration.

Here goes.

Why it takes me a fucking week to get anything done

The story starts when I set out to implement a simple HTTP route in POSSE Party that would return a JSON object mapping each post it syndicates from this blog to the permalinks of all the social media posts it creates. Seems easy, right?

Here's how it actually went. Each list item below represents a problem I pushed onto my mental stack in the course of working on this:

  1. Implement the feature in 5 minutes and 10 lines of code
  2. Consider that, as my first public-facing route that hits the database, I should probably cache the response
  3. Decide to reuse the caching API I made for Better with Becky
  4. Extract the caching functionality into a new gem
  5. Realize an innocuous Minitest update broke the m runner again, preventing me from running individual tests quickly
  6. Switch from Minitest to the TLDR test runner I built with Aaron last year
  7. Watch TLDR immediately blow up, because Ruby 3.4's Prism parser breaks its line-based CLI runner (e.g. tldr my_test.rb:14)
  8. Learn why it is much harder to determine method locations under Prism than it was with parse.y
  9. Fix TLDR, only to find that super_diff is generating warnings that will definitely impact TLDR users
  10. Clone super_diff's codebase, get its tests running, and reproduce the issue in a test case
  11. Attempt to fix the warning, but notice it'd require switching to an older way to forward arguments (from ... to *args, **kwargs, &block), which didn't smell right
  12. Search the exact text of the warning and find that it wasn't indicative of a real problem, but rather a bug in Ruby itself
  13. Install Ruby 3.4.2 in order to re-run my tests and see whether it cleared the warnings super_diff was generating

At this point, I was thirteen levels deep in this stack and it was straining my mental capacity. I ran into Becky in the hallway and was unnecessarily short with her, because my mind was struggling to keep all of this context in my working memory. It's now day three of this shit. When I woke up this morning, I knew I had to fix an issue in TLDR, but it took me a few minutes to even remember what the fuck I needed it for.

And now, hours later, here I am working in reverse as I pop each problem off the stack: closing tabs, resolving GitHub issues, publishing a TLDR release. (If you're keeping score, that puts me at level 5 of the above stack, about to pop up to level 3 and finally get to work on this caching library.) I needed a break, so I went for my daily jog to clear my head.

During my run, a thought occurred to me. You know, I don't even want POSSE Party to offer this feature.

Well, fuck.

My washing machine’s timer has been pegged at 1 minute remaining for over 45 minutes. With estimates like this, I should hire this thing as a software developer.

Announcing Merge Commits, my all-new podcast (sort of)

Okay, so hear me out. Last year, I started my first podcast: Breaking Change. It's a solo project that runs biweekly-ish with each episode running 2–3 hours. It's a low-stakes discussion meant to be digested in chunks—while you're in transit, doing chores, walking the dog, or trying to fall asleep. It covers the full gamut of my life and interests—from getting mad at technology in my personal life, to getting mad at technology in my work, to getting mad at technology during leisure activities. In its first 15 months, I've recorded 33 episodes and I'm approaching an impressive-sounding 100 hours of monologue content.

Today, I launched a more traditional, multi-human interview podcast… and dropped 36 fucking episodes on day one. It's called Merge Commits. Add them up, and that's over 35 hours of Searls-flavored content. You can subscribe via this dingus here:

Merge Commits artwork
Invite or suggest Justin as a guest!

Wait, go back to the part about already having 36 episodes, you might be thinking/screaming.

Well, the thing is, with one exception, none of these interviews are actually new. And I'm not the host of the show—I'm always the one being interviewed. See, since the first time someone asked me on their podcast (which appears to have been in June 2012), I've always made a habit of saving them for posterity. Over the past couple of days, I've worked through all 36 interviews I could find and pulled together the images and metadata needed to publish them as a standalone podcast.

Put another way: Merge Commits is a meta-feed of every episode of someone else's podcast where I'm the guest. Each show is like a git merge commit and only exists to connect the outside world to the Breaking Change cinematic universe. By all means, if you enjoy an interview, follow the link in the show notes and subscribe to the host's show! And if you have a podcast of your own and think I'd make a good guest, please let me know!

Look—like a lot of the shit I do—I've never heard of anyone else doing something like this. I know it's weird, but as most of these podcasts are now years out of production, I just wanted to be sure they wouldn't be completely lost to the sands of time. And, as I've recently discussed in Searls of Wisdom, I'm always eager to buttress my intellectual legacy.

Doc Searls (no relation) writes over at searls.com (which is why this site's domain is searls.co) about how the concept of human agency is being lost in the "agentic" hype:

My concern with both agentic and agentic AI is that concentrating development on AI agents (and digital “twins”) alone may neglect, override, or obstruct the agency of human beings, rather than extending or enlarging it. (For more on this, read Agentic AI Is the Next Big Thing but I’m Not Sure It’s What, by Adam Davidson in How to Geek. Also check out my Personal AI series, which addresses this issue most directly in Personal vs. Personal AI.)

Particularly interesting is that he's doing something about it, by chairing a IEEE spec dubbed "MyTerms":

Meet IEEE P7012, which “identifies/addresses the manner in which personal privacy terms are proffered and how they can be read and agreed to by machines.” It has been in the works since 2017, and should be ready later this year. (I say this as chair of the standard’s working group.) The nickname for P7012 is MyTerms (much as the nickname for the IEEE’s 802.11 standard is Wi-Fi). The idea behind MyTerms is that the sites and services of the world should agree to your terms, rather than the other way around.

MyTerms creates a new regime for privacy: one based on contract. With each MyTerm you are the first party. Not the website, the service, or the app maker. They are the second party. And terms can be friendly. For example, a prototype term called NoStalking says “Just show me ads not based on tracking me.” This is good for you, because you don’t get tracked, and good for the site because it leaves open the advertising option. NoStalking lives at Customer Commons, much as personal copyrights live at Creative Commons. (Yes, the former is modeled on the latter.)

How are the terms communicated? So MyTerms is expressed as some kind of structured data (JSON? I haven't read the spec) codification presented by the user's client (HTTP headers or some kind of handshake?), to which the server either agrees to or something-something (blocks access?). Then both parties record the agreement:

On your side—the first-party side—browser makers can build something into their product, or any developer can make a browser add-on (Firefox) or extension (the rest of them). On the site’s side—the second-party side—CMS makers can build something in, or any developer can make a plug-in (WordPress) or a module (Drupal).

Not answered in Doc's post (and I suspect, the rub) is how any of this will be enforced. In the late 90s, browser makers added a bold, green lock symbol to the location bar to convey a sense of safety to users that they were communicating over HTTPS. Then, there was a lucrative incentive at play: secure communications were necessary to get people to type their credit cards into a website. Today, the largest browser makers don't have any incentive to promote this. Could you imagine Microsoft, Google, or Apple making any of their EULA terms negotiable?

Maybe the idea is to put forward this spec and hope future regulations akin to the Digital Services Act will force sites to adopt it. I wish them luck with that.

Merge Commits artwork

Changelog: My Siri Theory

Merge Commits

Had a blast, as usual, joining my Changelog friends for a vigorous discussion of Apple's Intelligence struggles and the tumultuous state of the software industry.

It's also on YouTube:

Appearing on: The Changelog
Recorded on: 2025-03-18
Original URL: https://changelog.com/friends/85

Comments? Questions? Suggestion of a podcast I should guest on? podcast@searls.co

Though making things with computers involves a fair bit of misery and frustration, what keeps me at it is the feeling of FINALLY getting over the hump and unlocking the next chunk of rapid progress.

The closest analogue is how it felt to receive a present as a child and want to run away from the party to immediately play with it. As soon as I'm unblocked, I can't tear myself away.

Pro-tip: if your AirPods Pro have been in your ears for a while without audio playing and your device won't connect, brush the stem so as to adjust the volume. This will wake them up and they'll connect more quickly.

I wonder whether dudes enthralled by right-wing incel manosphere influencers realize that being an alpha male and having something to prove by posting online are fundamentally incompatible.

Tuesday, while recording an episode of The Changelog, Adam reminded me that my redirects from possyparty.com to posseparty.com didn't support HTTPS. Naturally, because this was caught live and on air and was my own damn fault, I immediately rushed to cover for the shame I felt by squirreling away and writing custom software. As we do.

See, if you're a cheapskate like me, you might have noticed that forwarding requests from one domain or subdomain to another while supporting HTTPS isn't particularly cheap with many DNS hosts. But the thing is, I am particularly cheap. So I built a cheap solution. It's called redirect-dingus:

What is it? It's a tiny Heroku nginx app that simply reads a couple environment variables and uses them to map request hostnames to your intended redirect targets for cases when you have some number of domains and subdomains that should redirect to some other canonical domain.

Check out the README for instructions on setting up your own Heroku app with it for your own domain redirect needs. I recommend forking it (just in case I decide to change the nginx config to redirect to an offshore casino or crypto scam someday), but you do you.

RevenueCat seems like a savvy, well-run business for mobile app developers trying to subscription payments in the land of native in-app purchase APIs. Every year they take the data on their platform and publish a survey of the results. Granted, there's definitely a selection bias at play—certain kinds of developers are surely more inclined to run their payments through a third-party as opposed to Apple's own APIs.

That said, it's a large enough sample size that the headline results are, as Scharon Harding at Ars Technica put it, "sobering". From the report itself:

Across all categories, nearly 20 percent reach $1,000 in revenue, while 5 percent reach the $10,000 mark. Revenue drop-off is steep, with many categories losing ~50 percent of apps at each milestone, emphasizing the challenge of sustained growth beyond early revenue benchmarks.

Accepted without argument is that subscription-based apps are the gold standard for making money on mobile, so one is left to surmise that these developers are way better off than the ones trying to charge a one-time, up front price for their apps. And only 5% of all of subscription apps earn enough revenue to replace a single developer salary for any given year.

Well, if you've ever wondered why some startup didn't have budget to hire you or your agency to build a native mobile app for them, here you go. Outside free-to-play games, the real money is going to companies that merely use mobile apps as a means of distribution and who generally butter their bread somehow else (think movie tickets, car insurance, sports betting).

Anyway, super encouraging thing to read first thing while sitting down to map out this subscription-based iOS app I'm planning to create. Always good to set expectations low, I guess.

I realize I'm a year late to dishing takes on Shogun, but since people keep recommending it, I thought I'd offer my 2¢ on a real problem I have with how it deals with spoken languages (and something I haven't heard anyone talk about anywhere else)