We're experiencing an extremely bizarre crash in our iOS/iPadOS app.
Some major changes in our latest release:
- Built with Xcode 13.0 on Big Sur 11.6
- Dropped support for iOS 13
- Updated Firebase to 8.9.1
- Built using the latest fastlane version
OK here's the bizarre part: our app works fine for users who download and install the app directly from the App Store or TestFlight on both iOS 14 & 15.
However, the app crashes on launch for iOS 14 users who get the app pushed to their phone by MDM from the App Store (public or private release, doesn't matter) and install it from a dialog box stating e.g. "AirWatch is about to install REDACTED from the App Store. Your iTunes account will not be charged for this app."
So what is the difference between launching the app after installing it yourself, and launching it after MDM installs it—in both cases, the app originates from the App Store?
Here is an example crash log:
{"app_name":"REDACTED","timestamp":"REDACTED","app_version":"REDACTED","slice_uuid":"REDACTED","adam_id":REDACTED,"build_version":"REDACTED","platform":0,"bundleID":"com.REDACTED.REDACTEDapp","share_with_app_devs":0,"is_first_party":0,"bug_type":"109","os_version":"iPhone OS 14.6 (18F72)","incident_id":"REDACTED","name":"REDACTED"}Incident Identifier: REDACTEDCrashReporter Key: REDACTEDHardware Model: iPhone11,8Process: REDACTED [1561]Path: /private/var/containers/Bundle/Application/REDACTED/REDACTED.app/REDACTEDIdentifier: com.REDACTED.REDACTEDappVersion: REDACTEDAppStoreTools: 13A1030dCode Type: ARM-64 (Native)Role: ForegroundParent Process: launchd [1]Coalition: com.REDACTED.REDACTEDapp [562]Date/Time: 2021-10-26 15:17:43.5707 -0500Launch Time: 2021-10-26 15:17:43.1390 -0500OS Version: iPhone OS 14.6 (18F72)Release Type: UserBaseband Version: 3.04.01Report Version: 104Exception Type: EXC_BAD_ACCESS (SIGSEGV)Exception Subtype: KERN_INVALID_ADDRESS at 0x0000000115596b28Termination Signal: Segmentation fault: 11Termination Reason: Namespace SIGNAL, Code 0xbTerminating Process: exc handler [1561]Highlighted by Thread: 0Backtrace not availableUnknown thread crashed with ARM Thread State (64-bit): x0: 0x000000016d11d5c0 x1: 0x0000000000000008 x2: 0x0000000000000000 x3: 0x0000000000000000 x4: 0x000000016d11d590 x5: 0x0000000000000020 x6: 0x736665727373616c x7: 0x0000000000000600 x8: 0x000000005f24901f x9: 0x000000000f8884a0 x10: 0x000000000000901c x11: 0x0000000000000003 x12: 0x0000000000ff0006 x13: 0x000000000000c000 x14: 0x0000000000000001 x15: 0xffffffffffffffff x16: 0x00000001030879cc x17: 0x6ae100016d11d440 x18: 0x0000000000000000 x19: 0x0000000115596b28 x20: 0x000000016d11daf8 x21: 0x000000016d11dad0 x22: 0x0000000105d00000 x23: 0x000000000f896b28 x24: 0x000000005f24901f x25: 0x0000000105d0e680 x26: 0x000000000000e688 x27: 0x000000016d11db08 x28: 0x0000000000001b99 fp: 0x000000016d11d6a0 lr: 0xba6bf2010306bf84 sp: 0x000000016d11d620 pc: 0x000000010306beec cpsr: 0x00000000 esr: 0x00000000 Address size faultBinary images description not availableError Formulating Crash Report:Failed to create CSSymbolicatorRef - corpse still valid ¯\_(ツ)_/¯EOFWe don't see these crashes appearing in Crashlytics or Xcode organizer, and we never get a stack trace or useful error message.
I can reproduce the crash locally using our UEM test account for VMWare Workspace One (AirWatch), but it does not happen on AdHoc or Developer builds, nor does it happen if Xcode is connected with a live debugger session.
We tried regenerating all our provisioning profiles with DER encoding, then re-signing and re-submitting the app to Apple, but it did not help the problem. We also tried turning off LTO (link-time optimization) and rolling back Firebase to the version before we had this problem 7.11.
Note: we added logging to everywhere we could think of in the app, including AppDelegate init methods, but when it crashes on iOS 14 MDM installs, we get no log messages at all in the console (whereas when it launches on iOS 15 we get plenty of log messages and no crash).
Here's another example crash log. Sometimes we get the VMRegion info like this, sometimes we don't.
Incident Identifier: B5916B4C-5F24-43B4-B038-8F20B4CF60DECrashReporter Key: a31b47951ef9c86f80d994b5128fd8f81060023aHardware Model: iPad6,8Process: REDACTED [514]Path: /Volumes/VOLUME/*/REDACTED.app/REDACTEDIdentifier: com.REDACTED.REDACTEDappVersion: 2021.0.58 (2021.44.0.433401)AppStoreTools: 13A1030dCode Type: ARM-64 (Native)Role: ForegroundParent Process: launchd [1]Coalition: com.REDACTED.REDACTEDapp [307]Date/Time: 2021-11-01 16:12:12.7748 -0700Launch Time: 2021-11-01 16:12:12.4122 -0700OS Version: iPhone OS 14.7.1 (18G82)Release Type: UserBaseband Version: 10.80.01Report Version: 104Exception Type: EXC_BAD_ACCESS (SIGSEGV)Exception Subtype: KERN_INVALID_ADDRESS at 0x000000011a3351c8VM Region Info: 0x11a3351c8 is not in any region. Bytes after previous region: 4553 Bytes before following region: 1395945016 REGION TYPE START - END [ VSIZE] PRT/MAX SHRMOD REGION DETAIL VM_ALLOCATE 11a330000-11a334000 [ 16K] rw-/rwx SM=PRV ---> GAP OF 0x53348000 BYTES Stack Guard 16d67c000-16d680000 [ 16K] ---/rwx SM=NUL Exception Note: EXC_CORPSE_NOTIFYTermination Reason: SIGNAL; [11]Terminating Process: exc handler [514]Terminating Process: exc handler [514]Highlighted by Thread: 0Backtrace not availableNo thread state (register information) availableBinary Images: 0x0 - 0xffffffffffffffff ??? unknown-arch <00000000000000000000000000000000> ???Error Formulating Crash Report:_dyld_process_info_create failed with 6Failed to create CSSymbolicatorRef - corpse still valid ¯\_(ツ)_/¯EOFAnother example:
Incident Identifier: 7C56CCB2-79BC-4AF2-91C2-5CFC8A0ABBA6CrashReporter Key: a31b47951ef9c86f80d994b5128fd8f81060023aHardware Model: iPad6,8Process: REDACTED MDM [27056]Path: /Volumes/VOLUME/*/REDACTED MDM.app/REDACTED MDMIdentifier: com.REDACTED.REDACTEDappmdmVersion: 2021.0.63 (2021.46.5.000005)AppStoreTools: 13A1030dCode Type: ARM-64 (Native)Role: ForegroundParent Process: launchd [1]Coalition: com.REDACTED.REDACTEDappmdm [1967]Date/Time: 2021-11-11 17:22:03.7594 -0800Launch Time: 2021-11-11 17:22:01.0685 -0800OS Version: iPhone OS 14.7.1 (18G82)Release Type: UserBaseband Version: 10.80.01Report Version: 104Exception Type: EXC_BAD_ACCESS (SIGSEGV)Exception Subtype: KERN_INVALID_ADDRESS at 0x0000000180000fa8VM Region Info: 0x180000fa8 is not in any region. Bytes after previous region: 4009 Bytes before following region: 685764696 REGION TYPE START - END [ VSIZE] PRT/MAX SHRMOD REGION DETAIL VM_ALLOCATE 17fffc000-180000000 [ 16K] rw-/rwx SM=PRV ---> GAP OF 0x28e00000 BYTES unused shlib __TEXT 1a8e00000-1b0000000 [114.0M] r-x/r-x SM=COW ... this processException Note: EXC_CORPSE_NOTIFYTermination Reason: SIGNAL; [11]Terminating Process: exc handler [27056]Terminating Process: exc handler [27056]Highlighted by Thread: 0Backtrace not availableNo thread state (register information) availableBinary Images: 0x0 - 0xffffffffffffffff ??? unknown-arch <00000000000000000000000000000000> ???Error Formulating Crash Report:_dyld_process_info_create failed with 6Failed to create CSSymbolicatorRef - corpse still valid ¯\_(ツ)_/¯EOFThe only common thread seems to be that there's a VM_ALLOCATE that allocated a block of size 0x4000 bits. That's a pretty common size for a DATA segment so I don't think it's particularly useful information.
Update:
We've discovered that setting the deployment target back down to iOS 13 makes the problem not happen. It only happens when the deployment target is 14 or higher. Also, the crash happens before we even run a single line of code in our app, according to not seeing any os_log messages that we added to all the init methods in our UIApplication and UIApplicationDelegate classses and many static load methods.
