{"id":"019d3e94-a64c-7fcc-86fc-e5216e4417ee","title":"The Wrong Boundary","slug":"2026/03/the-wrong-boundary","renderedHtml":"<p>It's pretty easy to find people in IRC who've seen new programmers blame everything they can, when something goes wrong: it's how console I/O is broken, the library is wrong, the database is wrong, the pointers are wrong, the JVM has a bug that's been sticking around for twenty years. Meanwhile, the more experienced developers sigh and point out the wrong strategy the code uses, and the fix is easy once the paradigms are corrected. One of the hallmarks of being a junior developer is to look everywhere but the mirror.</p>\n<p>But the experts make similar mistakes: experts tend to be better about boundary conditions than junior developers, but they suffer the same problems: by prematurely defining where the problems <em>can</em> be, we miss where the problems <em>are</em>.</p>\n<h2>Is it in the room with us right now?</h2>\n<p>Junior programmers mess up paradigms because they think they're more experienced than they actually are. They read something from an input stream, for example, and think &quot;Well, I read it&quot; - without realizing the input stream has more to read, corrupting their future reads unless they clear out everything they <em>don't</em> want - an error most experienced programmers <em>remember</em> as a classic error.</p>\n<p>They end up blaming the system for not working the way they think it should. To some degree, perhaps they're right - their assumptions have merit, it's just that someone has to <em>make</em> the systems work the way the programmers expect, and there are often <em>concrete reasons</em> the system doesn't do that already.</p>\n<p>Those junior programmers are struggling with reality and understanding; they trust themselves too much and the system too little.</p>\n<p>ByteCode.News is a connected system; this website is connected to IRC, Slack, and Discord as well as the web. In the IRC adapter, I just spent three days worrying over the speed of queue processing; the adapter was sending responses back every <em>two seconds</em> or so, which is impossibly slow when the internal message queue processed each message in around 4ms. I tried adaptive delays, I tried input throttle measurements, I measured throughput to see if the production system just had something configured incorrectly.</p>\n<p>It was all wasted effort. It was <em>me</em>. A classic &quot;ID10T&quot; error - my <em>irc client</em> was throttling outgoing messages. <em>I was sending messages at a rate of one every two seconds</em> without realizing it, since my client showed them all to me instantly and timestamped them <em>when I queued them</em> and not when they were sent.</p>\n<p>I observed poor throughput, and because I <em>did</em> trust the system, I assumed my code was wrong, and I wasted time trying to find the error so I could fix it. I trusted the wrong thing, just like a junior programmer: I assumed my tests were wrong, my results were wrong, that <em>I</em> was wrong, and I was - but the error was in the <em>location</em> of the problem.</p>\n<p>A single IRC chat client configuration cleared everything up.</p>\n<blockquote>\n<p>The setting, if you're interested, is from <a href=\"https://weechat.org/\" title=\"an extensible terminal IRC client with a plugin architecture, scriptable in Python, Perl, Ruby, and others.\">weechat</a>: <code>irc.server.libera.anti_flood</code>.</p>\n</blockquote>\n<h2>What the measurement wasn't saying</h2>\n<p>The <em>error observation</em> wasn't incorrect; the time <code>nevet</code> took to respond to my instructions <em>was</em> slow. It was my scoping that was wrong, because I didn't know what (or whom) to ask. The data was there; <code>nevet</code> has logs that actually did record the incoming time, I just never thought to scan the logs in detail for the <em>input</em>, only the <em>output</em>.</p>\n<p>It's a subtler version of the junior programmer mistake. &quot;The JVM is broken&quot; is at least falsifiable - you can point the junior dev at the test kit, or a body of work on the subject they're struggling with. But &quot;my adaptive sender isn't working well enough yet&quot; is a hypothesis that appears to engage with the evidence, while actually just reasserting the original attribution. I was measuring inside a boundary I'd drawn poorly, and the measurements couldn't tell me that, because they couldn't see outside it.</p>\n<p>It's &quot;show me where in this circle the problem exists,&quot; while the problem is the eraser on the pencil drawing the circle.</p>\n<h2>Questions Shape Their Answers</h2>\n<p>Programmer Barbie says &quot;problem definition is hard!&quot; and she's not wrong. Problem definition <em>is</em> hard, because every one of us has assumptions about relevant experience and our environment. We say &quot;what's happening&quot; and get the view from 10000 feet - from a specific observer, maybe ourselves, and that can fail to account for... almost anything. If you're not used to thinking about how many packets a record takes on the wire, then you're not going to think that 1501 bytes can take twice as long as 1500 bytes to transmit.</p>\n<blockquote>\n<p>This is related to <a href=\"https://en.wikipedia.org/wiki/Maximum_transmission_unit\" title=\"the Maximum Transmission Unit - the largest packet size a network layer will transmit without fragmentation. For Ethernet, this is typically 1500 bytes; exceeding it splits the payload across multiple packets.\">MTU</a> - the packet size for Ethernet traffic. Once you go over the MTU size, you have a new packet. The maximum packet size is 1500 bytes, so if you send 1501 bytes in a single network transaction, you have to send two packets instead of one, for that extra byte.</p>\n</blockquote>\n<p>Once you <em>know</em>, it's easy to fix: send the 1500 bytes, only, unless you have no other choice! Look for what you can optimize; it's usually there. (This is why <a href=\"https://json.org/\" title=\"&lt;reply&gt;JSON is the JavaScript Object Notation standard, a specification for human-readable structures, based on how JavaScript represents objects in programs. It's fairly terse, at least moreso than XML, but is far more free-form.\">JSON</a> usually wins over <a href=\"https://www.w3.org/XML/\" title=\"&lt;reply&gt;XML stands for &quot;Extensible Markup Language,&quot; and is a variant of SGML; XML is a markup language that allows documents to be strictly or loosely specified, verifiable, and heavily structured. It's well known for being *incredibly* verbose compared to most object notation languages, and is not an object notation language itself.\">XML</a> in wire performance: it's often <em>smaller</em>... and why <a href=\"https://protobuf.dev/\" title=\"&lt;reply&gt;Protocol Buffers, Google's binary serialization format, is more compact and faster to parse than JSON or XML, at the cost of human readability. Widely used in gRPC and high-throughput systems.\">protobuf</a> wins over JSON. It's all a tradeoff of verifiability and readability and memory consumption.)</p>\n<p>The hard part of all of this is <em>recognition</em>. It's never going to be free; I was doing the problem analysis with AI agents, trying to find that external perspective that would clear it all up, and not one of them mentioned the problem scope, because I declared the problem and the LLMs assumed I knew more of what I was talking about than I did.</p>\n<p>Once I explicitly widened the scope and asked the right question, the fix was 30 seconds away.</p>\n<h2>Locate first</h2>\n<p>The discipline this points to is simple to state and easy to skip: locate the problem before you characterize it. <em>Really</em> locate it. The problem isn't &quot;my code is slow&quot; - where, specifically, is time being spent? Have you measured? Do the measurements meet your expectations in each environment? Why?</p>\n<p>That's a story from the past, too: a test server and a production server were both running a scenario in the same amount of time, despite the test server being <em>vastly</em> constrained compared to the production system. The problem was in <em>database colocation</em> - the test server was running DB2 on the same machine that WebSphere was running on, and the production server was running WebSphere in Connecticut and the database was in the Midwest - the time being lost was <em>network traffic</em>, quite literally. Yet the problem statement was &quot;why are we paying for a killer prod system when the test system runs as well?&quot;</p>\n<p>The stories we come in with affect everything we do, something that affects humans in our entire lives. The junior dev's story is that the environment is hostile; the senior engineer's story is that their own code is the most likely culprit, because that's the story both have lived. Both stories feel like reasonable priors. Both can cause you to instrument around the actual problem rather than toward it.</p>\n<p>Your observation tells you something is wrong. It doesn't tell you what. Treating those as the same thing is where the time goes.</p>","excerpt":"Junior programmers mess up paradigms because they think they're more experienced than they actually are. They end up blaming the system for not working the way they think it should. I trusted the wrong thing, just like a junior programmer: I assumed my tests were wrong, my results were wrong, that I was wrong, and I was - but the error was in the location of the problem.","authorId":"019c5c8a-609d-7cd4-975b-50bbcc412a33","authorDisplayName":"dreamreal","status":"APPROVED","publishedAt":"2026-03-30T11:51:06.473Z","sortOrder":0,"createdAt":"2026-03-30T11:50:22.540185Z","updatedAt":"2026-03-30T11:51:06.548518Z","commentCount":0,"tags":["debugging","methodology","performance"],"categories":[],"markdownSource":null}