Rebuild note

The release build is not the app you tested

Two bugs that could not be reproduced in development, because the tools that caused them do not run in development. Both shipped. Both are the same mistake.

What this is based on Two release-only failures in StudyPomo, and the keep rules that are still in the shipped 1.0.0 mapping.
In this piece
  1. A release build is a different program
  2. The crash that only existed in release
  3. The sounds that vanished
  4. Why neither could be reproduced
  5. What we changed about how we work
  6. What we are not telling you

StudyPomo went to Google Play as version 1.0.0, build 16. A fair share of those sixteen went on one class of problem: the app worked perfectly on the bench and was broken the moment it was built the way users would receive it.

Both failures below were real, both reached a release artefact, and neither could be reproduced in development. That is not bad luck. It is a property of how Android release builds are made, and once you see it the class of bug becomes predictable rather than mysterious.

A release build is a different program

Running the app from your machine produces a debug build. Shipping produces a release build, and three things happen in release that never happen in debug.

Code shrinking removes classes and methods nothing appears to call. Obfuscation renames what is left. Resource shrinking removes drawables, sounds and layouts nothing appears to reference. On modern Android the first two are R8, and the third is a separate flag.

Every one of those is a compiler deciding what your program does not need, from a static reading of the code. So the artefact you tested is not the artefact you uploaded, and anything the compiler cannot see is a candidate for deletion.

The crash that only existed in release

The first symptom was the worst kind: the release build crashed on launch. Not on a particular screen, not after a particular action. It started and stopped.

The cause was that R8 had removed a class generated by WorkManager, which is reached by reflection rather than by a direct call. A static analyser reads the code and asks who calls this. When the answer is nobody, because the only caller constructs the name as a string at runtime, the class is dead weight and gets stripped. The app then asks for it by name and finds nothing there.

The fix is a keep rule, which is an instruction telling R8 that something must survive whatever the analysis concludes. Ours live in android/app/proguard-rules.pro and cover WorkManager and androidx.startup. You can confirm they did their job on the shipped build: the 1.0.0 mapping file still contains androidx.work.impl.WorkManagerImpl and the startup classes, which is what a keep rule looks like from the outside.

The general shape is worth remembering, because it recurs. Anything loaded by name, by annotation, or by a library’s own reflection is invisible to the compiler. Plugins are a common source, because a plugin’s registration is frequently generated and frequently reflective, and you did not write the line that would have made the reference visible.

The sounds that vanished

The second one was quieter and took longer to understand. In a release build, every notification sound and the notification icon were gone. The timer ran, the session ended, and nothing happened.

Resource shrinking removes resources that nothing references. StudyPomo is written in Flutter, so its sounds and icons are referenced from Dart. The shrinker reads Java and Kotlin. From where it was standing, those files were unused, so it did what it was asked to do and removed them.

There are ways to fight this resource by resource, with a keep file that lists what must survive. We did not take that route. isShrinkResources is off, deliberately, and there is a comment in the build file saying so, because the failure is silent and the next person to turn it back on will not find out until a release build reaches a phone.

That is a real trade and worth stating plainly. Leaving it off means shipping an APK larger than it strictly needs to be. We do not have a measured figure for how much larger, and rather than estimate one, we will say that the alternative was a timer that ends a session in silence, and no plausible saving is worth that.

Why neither could be reproduced

This is the part that generalises past Android.

Both bugs were introduced by tools that only run when you are not looking. The whole point of a debug build is a fast loop, so it skips the expensive passes. That is the right default and it produces a specific blind spot: the pipeline that creates your bugs is the one you never exercise until release.

Which means the ordinary discipline of testing does not reach these. You can be thorough, methodical and completely diligent against a debug build, and learn nothing at all about the artefact you are about to upload.

What we changed about how we work

Install the release artefact on a real device before every submission. Not the debug build, not a profile build. The exact file that would go to Play, installed the way a user would receive it. This is the only step that catches this class at all, and it takes minutes.

Treat a keep rule as part of adding a dependency. If a library does anything reflective, its keep rules go in when the library goes in, not after a crash report. Most maintained libraries document theirs, and many ship them automatically.

Write down why a flag is off. isShrinkResources being disabled looks like an oversight to anyone reading the build file fresh. Left unexplained, someone eventually tidies it up, and the sounds disappear again on a build nobody connects to that change.

What we are not telling you

We do not know whether either bug reached a user, because they were both caught before 1.0.0 went out. The app has no analytics and no crash reporting, which is a choice we are content with and also a limit we should name: we would not have learned about the launch crash from the field. We learned about it because somebody installed the release build on a phone and it died.

That is the entire recommendation, really. The release build is not the app you tested, so test the release build.

Written by Adeboye Oluwatimileyin and Moses-Azuoru George, the two people who are Emberfig. We make money from AdMob inside the apps. This site carries no ads yet. You never pay us and we never sell your data.