# QA automation

> This is my day job. It also yields the least — because I'm usually already standing in the same place. And of the 4.5 techniques I wrote down as adopted, only one was in when I checked, so I put in the other 3.5 over the next two days (2026-09-28–29). The sixth, clumsy, I took apart twice and have yet to run once (2026-10-02).

- Headline number: 6 written · 4.5 of 7 applied
- Rendered page: https://aitryed.com/en/teardowns/qa/
- Other language: https://aitryed.com/ko/teardowns/qa/index.md
- Site guide for agents: https://aitryed.com/en/llms.txt

---

**This is my day job. It is also the shelf that has taught me the least — and that turns out not to be a bad thing.**

First, what "QA automation" is.

When you build a game or an app, someone has to **press everything and check it works.** Press a button, see if the right screen comes up, confirm the value landed. A person doing that by hand is QA. **Getting a machine to do the pressing** is QA automation.

There are six pieces here, each one someone else's automation, opened up.

## My own field, and the smallest harvest — which is not a complaint

Across the first five pieces, the techniques I **wrote down as adopted** number **4.5**. Fewer than any other shelf. (Only one had actually gone in when I checked — see the 2026-09 check below.)

My first thought was *I must be reading them badly.* That wasn't it. **Hold the same problem long enough and everyone arrives at similar answers.**

Put it this way: when there are only a few routes up a mountain, the person who climbed first and the person who climbed later meet at the same places. At that point the finding is not *nothing to learn* — it is **my route was right.**

## Why the research still pays when nothing transfers

This is the important part of the shelf. Two different kinds of return came out.

**One — a conclusion I held alone became a conclusion held by two.** Some judgements I had made on my own, with **no way to know whether they were right.** Then someone I have never met, at a different company, **had arrived independently at exactly the same answer.** A sample of one became a sample of two. Even with zero techniques transferred, that is worth a lot.

**Two — someone else's principle became a ruler for my own thing.** In one piece the transferable technique count was **zero.** But taking the principle they had set and measuring my code against it, I found **a defence I had built was not wired into any actual run.** Nothing learned, and something to fix.

## "Already there" did not always mean the same thing

At two pieces I read it as one thing: *I already arrived at this.* At four, **two different states** were hiding under that phrase. Having arrived at the same answer and **having written the answer down without ever doing it** are not the same.

And I was caught by the second one — **something I believed I had done turned out to be something I had written down as intending to do.**

## One permission that changed the nature of the evidence

There is a harvest here that isn't a technique.

Checking whether a game's drop rate matches what was advertised used to mean **pulling hundreds of times and doing statistics.** Slow, and the answer comes out as *probably right*.

When access to the underlying data arrived, the same check became **one query, five seconds.** And the answer stopped being *probably* and became **certain.**

→ What I took: **it wasn't a faster tool. The vantage point changed, and that changed the kind of evidence available.**

## In the fifth, I went to judge a detector and got caught by it

The first four were other people's talks and articles. The fifth was **the first time I pulled someone's code and ran it against my own data.**

I went to test whether their detector caught what it claimed. **Run against my data, it flagged a pile of things that were fine.** Not because the tool is bad — because it was **the wrong tool for the place I meant to put it.** Without measuring, it would have come straight in.

## Of the 4.5 techniques I wrote down as adopted, one reached the code — a 2026-09 check

About a month on, I opened my code to see whether what each piece said it would take had actually gone in.

| Piece | What it said I'd take | As of 2026-09 |
|---|---|---|
| [Asleep's QA automation](/en/teardowns/qa/asleep-qa/index.md) | Count passes that came from a rough coordinate tap separately | **In** — a tap ledger that records the evidence behind every tap |
| [An autoplay test tool](/en/teardowns/qa/autoplay-test-tool/index.md) | Catch "pushing but not moving" as being stuck · confirm only when two clocks ring together | **Not in** at the check → **both went in** (9/28) |
| [watermarks-remover](/en/teardowns/qa/watermarks-remover/index.md) | One sentence-rhythm term · a check that stops on an unknown kind of file | **Neither was in** at the check → **both went in**: the stopping check (9/28) and the rhythm term (9/29, long-form only) |
| [Toss Nebula](/en/teardowns/qa/toss-nebula/index.md) | Fix the "gateway that exists twice" it surfaced instead of a technique | **Half** — the real run path now has a lock; the gateway still exists twice |
| [A talk on QA infrastructure](/en/teardowns/qa/qa-infrastructure/index.md) | Use the gacha case as an example in writing | **In** — ["Probably right" is not a verdict](/en/writing/verdicts/evidence-grades/index.md) |

**Writing "adopt" and putting it in turned out to be separate jobs.** Marking a row "adopted" in a verdict table feels like the work is done at that moment, and the code knows nothing about it. So this shelf's number is now written as **"4.5 of 4.5 applied"** rather than "4.5 techniques" — the one that was in at the check, plus the 3.5 put in over the next two days (a publish check that stops on an unknown kind of file · the contradiction check for being stuck · the two-clocks confirmation · a long-form-only sentence-rhythm check). The effects differ. The rhythm check caught two easy-layer passages on its first run and they were fixed; the two stuck-detection pieces haven't filtered anything in simulation yet — the current bot pushes the whole time it walks, and nothing in the simulation inches forward — so what went in and what has had an effect are counted separately too.

One mistake ran the other way. In the autoplay piece I said my stuck detection had only one layer; opening the code, **a second layer was already there** — that piece now says so.

And outside this shelf, something built by someone else entered my QA pipeline for the first time: the evaluation model **Jev**, attached as the photo judge's "second eye" (2026-09-24). It came in through a four-day trial rather than a teardown, and in the range measured it has not yet caught a real error → [The photo judge](/en/built/android-qa/judge/index.md).

## The sixth piece took the same subject apart twice and never ran it

[clumsy](/en/teardowns/qa/clumsy/index.md) is a tool that makes a PC's internet connection worse on purpose. I opened it for disconnect testing, and only after finishing did I find **a report from August that had already reached the same conclusion.** Both times I wrote "use it". Neither time did I run it.

The check above taught me that writing something down and putting it in are separate jobs; this time I went through that twice inside a single subject. So the shelf's number becomes **"4.5 of 7 applied"** — clumsy adds 2.5 to what was written down, and nothing to what went in.

## The detailed record starts here

Where other people's test automation gets taken apart: device farms, unattended regression on real hardware, game autoplay tools, agent-driven test platforms.

**It's my day job and it yields the least.** Across five pieces, 4.5 techniques transferred (as of 2026-09-29: 4.5 written into verdict tables · 4.5 in the code — 1 at the check, 3.5 after · see the 2026-09 check above). Not because the teardowns were bad — because enough years under the same constraints tend to land you in the same place already.

## Which changes what this bucket is worth

| Subject | Techniques | What actually stayed |
|---|---|---|
| [Asleep's QA automation](/en/teardowns/qa/asleep-qa/index.md) | **1** of 11 | That under the same constraints **someone else arrived independently at the same decisions** |
| [Toss Nebula](/en/teardowns/qa/toss-nebula/index.md) | **0** | Using their principle as a ruler showed that **my own gateway exists twice** |
| [An autoplay test tool](/en/teardowns/qa/autoplay-test-tool/index.md) | **2** of 4 | Catch being stuck as **a contradiction in the data**, and clear false positives with **an AND over two timers** |
| [A talk on QA infrastructure](/en/teardowns/qa/qa-infrastructure/index.md) | **0** | Not a tool but a case where **the character of the evidence** is decided by access |
| [watermarks-remover](/en/teardowns/qa/watermarks-remover/index.md) | **1.5** of 4 | Measure a detector's false-positive rate **on your own corpus** before adopting it — and on the corpus it will actually run against |
| [clumsy](/en/teardowns/qa/clumsy/index.md) | **2.5** of 6 (0 applied) | Analysing the same subject twice left the verdict exactly where it was — **only a measurement moves a verdict** |

An investigation that yields nothing isn't wasted. Asleep **turned a set of conclusions with a sample size of one into two** — on-prem, fail-closed on live, unattended regression, element-based targeting. Nebula handed over zero techniques and instead surfaced that **the official gateway's defences aren't wired into the path a real run takes.**

## And in both of the early ones, my own verdict got overturned

Asleep took two corrections. I wrote down "silence detection is a new harvest," then deleted it because **it was already in my code**; then tried to close at "zero new," and found that **my excerpt had been too thin** — the section the author calls the core wasn't in it at all.

Nebula lost the only harvest in its first draft. I'd adopted an in-device controller on the strength of a real measurement — `uiautomator dump` at 2,350 ms — and then found **that call appears zero times in my pipeline.** A game client draws the whole screen into one native surface, so there is no accessibility tree to dump.

> The measurement was right and the target was wrong. Before benchmarking anything, check **whether the call actually happens in my run.**

## At four pieces, "already there" split into two meanings

At two I read it as one thing — *"I've already reached the same conclusion."* At four, **two different things turned out to be mixed together inside that.**

**One is genuinely the same place** — ten of Asleep's eleven decisions were that. **The other looked like the same place and was one layer shallower.** The autoplay piece was that. I was also judging when a bot is stuck, and I wasn't wrong, **I had simply never separated the layer that produces candidates from the layer that removes false positives.**

**You cannot tell these two apart without opening the thing**, because both look like *"we already do that."* The moment I split the layers, **the cause of my false positives acquired a name** — *"a normal state where it stopped on purpose."*

**(Correction, as of 2026-09)** "Never separated the layers" was wrong for the autoplay piece. Since the July code, my stuck detection has had **a second layer that confirms by trying to escape**, behind the candidate (under 60 cm of movement in 3 seconds). What was one layer shallower is not the number of layers but **the contradiction check that looks at input and movement together.** That went in on 2026-09-28.

## And this bucket holds one harvest that isn't a tool

The infrastructure talk yielded zero techniques and left **one case: verifying gacha probability.** What used to be hundreds of pulls and **a statistical estimate** became **one query in five seconds** once read access existed — and **the character of the evidence changed from estimate to certainty.**

It didn't get faster. **What counts as evidence** changed. It goes straight into [the writing about verdicts](/en/writing/verdicts/index.md), which makes it **the only harvest from this bucket that turns into prose.** (As of 2026-09 it went in: ["Probably right" is not a verdict](/en/writing/verdicts/evidence-grades/index.md) uses this case as a jump of several rows "from a sample to a direct read of the source".)

## The fifth was this bucket's first repository, and I went to judge a detector and got caught by it

The first four were all somebody's talk or article. [watermarks-remover](/en/teardowns/qa/watermarks-remover/index.md) is **the first piece here where I read code**, which let me do the one thing only that allows — run someone else's detector **against my own data.**

The result was the most expensive thing this bucket has produced. **A metric built to separate AI writing flagged, on its first outing, something a human wrote — me.** Sentence-length coefficient of variation across 98 long pieces on this site sits at a comfortable median of 0.668; move it to the drafts from the channel where the rule would actually live, and one in six falls under the threshold — **and that one was already published.**

> Measure a false-positive rate **in the wrong place** and the rate itself is a lie. Calibrate on long-form, apply to short-form, and you are worse off than before you measured.

Read as QA this is a familiar failure: **test data that doesn't match the production distribution.** Except this time I was the one who chose the test data. The Nebula piece taught me to ask *"does this call actually happen in my runs"*; this was **the hole one seat over** — the call does happen, but the sample I measured came from somewhere else.