You can cancel the autopay after the in-flight payment fully settles, but not before. ACH payments typically take 3-5 business days to settle.
Once the payment is done, log into your landlord dashboard at rent.app ("Split Pay by Rent App") and cancel the autopay there.
Note: This applies to tenants who pay you directly through Split Pay. Portal autopays follow your portal's own rules and timelines.