← All guides

MsgId and EndToEndId: the uniqueness rules that trip up ERP exports

How the two identifiers in a pain.001 file are supposed to work, why duplicates cause whole-file rejections and reconciliation chaos, and how to generate identifiers that never collide.

Every pain.001 file carries name tags. The file itself wears one, called MsgId, in its group header. Every payment inside wears one too, called EndToEndId. These identifiers look like bureaucracy — short strings nobody reads — until the day they are wrong, when they become the most expensive strings in the building. A reused MsgId can make a bank refuse an entire payroll file as a suspected double submission. A duplicated EndToEndId can make two different suppliers’ payments indistinguishable to your reconciliation software, so one gets matched and the other haunts the suspense account for a quarter. This guide explains what each identifier is for, the uniqueness each one needs, the ways ERP exports get them wrong, and how to generate them so the problem class disappears.

Two identifiers, two different jobs

MsgId, the message identification, names one file — more precisely, one message — as sent by the initiating party. Its audience is the bank’s intake system. Banks keep a memory of the MsgIds they have received from you, and that memory is a safety feature: if the same message arrives twice, the bank suspects the classic accident of payment operations, the double upload, and protects you by refusing the second copy. The EndToEndId has a longer journey in mind. It is assigned by the debtor to one payment and is designed to travel with that payment through every system to the creditor and back again in status reports and statements, unchanged. When a payment is returned, the return references the EndToEndId, which is how your software knows which original payment just came back. MsgId guards the door; EndToEndId follows the payment for its whole life.

There is a third identifier you will see nearby, PmtInfId, which names each payment block inside the file, and identifiers like InstrId that a sending system may add for its own tracking. They matter for housekeeping, but the two with scheme-level uniqueness expectations are MsgId and EndToEndId. Both are limited to 35 characters and must live inside the SEPA character set — letters, digits and a little punctuation — which connects this topic to the character set rules: an identifier built from a timestamp with colons, or a reference with a hash sign, can break the set rules before uniqueness is even considered. Keep identifiers boring: letters, digits and hyphens are all you need.

The uniqueness each identifier actually needs

For MsgId, the safe rule is: unique per sender, forever, from the bank’s point of view. Banks differ in how long they remember and how strictly they enforce, and their guidelines say things like the identifier must not repeat within a given period. Do not engineer to the loosest reading. If your MsgId has never been used before and will never be used again, every bank’s duplicate check passes trivially. For EndToEndId, the scheme expectation is uniqueness per payment — in practice, within the file at minimum, and within your own books across time if you want reconciliation and returns handling to work without human detective effort. A return that references an EndToEndId you used forty times is a riddle, not a reference. The standard even provides an escape value, NOTPROVIDED, for cases where no end-to-end reference exists — use it deliberately if you must, knowing that every payment stamped NOTPROVIDED is a payment your future self cannot trace automatically.

How ERP exports get them wrong

The failure patterns repeat across systems. Pattern one: the constant. A template stamps the same MsgId on every export — the company name, the word PAYMENTS, a project code from the implementation era — and the bank accepts the first file and refuses every later one, or accepts them until a guideline tightens and then refuses. Pattern two: the daily counter. Identifiers built from the date alone, or date plus a counter that resets when the export job restarts, collide whenever two files are produced in one day or a failed run is re-exported. Pattern three: the meaningful reference. EndToEndId filled with the invoice number sounds sensible and works until two invoices from one supplier share a number across years, or a credit note reuses the invoice it corrects. Pattern four: the retry. An upload times out, the operator exports again — new file, same identifiers, because the export is deterministic — and the bank, having received the first attempt perfectly well, refuses the second as a duplicate, which the operator reads as the bank losing the file.

Pattern five deserves its own paragraph because it fools careful people: the test file that reached production identifiers. During implementation, test exports use a small range of tidy identifiers. Months later the production sequence grows into the same range, or the test bank connection and the production connection share a duplicate memory during a migration, and suddenly live files collide with ghosts. The defence is namespaces: make the identifier itself carry its world with it — a prefix for the system and environment is cheap insurance — and never let two systems generate from the same scheme without a distinguishing component. Identifiers are the cheapest place in the whole payment architecture to buy uniqueness; spend the characters.

A generation scheme that never collides

• Build the identifier from parts that are unique in combination: a short system prefix, the date, and a sequence number drawn from a counter that only ever increases.
• Store the counter outside the export job — in the database, not in a file the job can lose or a variable that resets on restart.
• Never reuse an identifier for a corrected file: a regenerated export is a new message and gets a new MsgId, even if its content is identical.
• For EndToEndId, anchor it to the payment’s permanent identity in your system — the payment record’s own key — not to a document number that can repeat.
• Keep to letters, digits and hyphens, and stay well under 35 characters so no system truncates two different identifiers into the same string.
• Log the identifiers with the run: when a bank asks about a file, the MsgId is how you will both find it in seconds.

A detail that saves arguments with banks: when you contact support about a rejected or missing file, quote the MsgId and the creation timestamp before anything else. It is the key their intake logs are indexed by, and the conversation shortens from archaeology to lookup. Internally, the same habit pays off when files are validated before upload: our reports carry the file’s MsgId and list any EndToEndId that appears more than once, with the payments involved, so a duplicate is a line in a report rather than a discovery at the bank. Duplicate references are also one of the standing entries in our overview of why banks reject SEPA files, because the pattern is so consistent across exporters.

Uniqueness windows: how long is long enough?

A fair question follows: must a MsgId be unique for all time, or only within some window? Bank guidelines phrase it as a window — the identifier must not repeat within a stated period, which can be days, months, or the lifetime of the account relationship, and differs by bank and channel. The temptation is to design to the shortest window you can find and reclaim tidy short identifiers sooner. Resist it, for two reasons. First, windows are bank policy, and policy changes: a bank that remembered 90 days can extend to a year in a guideline update, and your three-month rotation quietly becomes a collision generator. Second, your own systems have longer memories than any bank window: archives, audits and dispute investigations reach back years, and an identifier that names two different files in your own history is a trap you built for yourself. Forever-unique costs almost nothing — a date plus an ever-increasing sequence fits in 35 characters with room to spare — while window-unique saves nothing worth having. Design for forever, and every bank’s window, present and future, is satisfied as a side effect.

EndToEndId deserves the same generosity. Within one file, uniqueness is the floor, not the goal: the identifier’s real work begins after the file is accepted, when status reports, returns and statement lines come back referencing it, sometimes weeks later, occasionally after a repair or a recall months down the line. If the same EndToEndId names a salary payment in March and a supplier payment in October, the October return that references it has two candidate originals, and a human must disambiguate with amounts and dates — exactly the manual work the identifier existed to eliminate. Anchoring EndToEndId to the payment record’s permanent key in your system, as the checklist recommends, makes cross-time uniqueness automatic, because database keys already never repeat. Systems that generate EndToEndId from business documents — invoice numbers, payroll periods, order references — should treat collisions as a data smell worth one extra field: the document reference can ride in the remittance information, where repetition is harmless, while the identifier stays unique. Each datum in its own field, each field doing its own job — the recurring theme of every rule in this series.

There is one more collision hiding in plain sight: truncation. Identifiers may be up to 35 characters, but not every system in the chain stores all 35, and a receiving or intermediate system that quietly shortens a long identifier can turn two distinct values into one identical stored string — the uniqueness you generated, destroyed downstream by a field length you never saw. The defence is to stay boringly short: an identifier of 20 characters carries a system prefix, a date and a nine-digit sequence with room to spare, and survives every realistic storage limit in the chain. Length is not ambition. If your scheme needs more than 35 characters to be unique, the scheme is wrong; if it needs more than about 24, it is probably carrying information that belongs in a data field instead of an identifier. Identifiers should be keys, not messages: opaque, short, and unique — everything else about the payment already has a proper home somewhere in the file.

And a human note to end the technical part: when a duplicate rejection lands, the operator who re-sent the file is usually blamed, and usually should not be. They were working with a system that gave them no way to see whether the first attempt arrived and no warning that the retry would look like a duplicate. Fix the visibility — a run log keyed by MsgId, an upload status the operator can check — and the behaviour fixes itself. Identifiers are where system design and human habit meet, and they fail at the seam when either side is left to guess.

The deeper lesson is that identifiers are a design decision, not a formatting detail. Teams decide them once, usually during implementation, usually in a hurry, and then live inside the decision for a decade. Decide as if the file count will grow, systems will be replaced, and an operator will one day re-export under pressure at month-end — because all three will happen. A validator can catch the duplicates that slip through: run your file through the free check and duplicated EndToEndId values are listed explicitly. For teams exporting from several systems, the batch checks on paid plans sweep every file in the run for internal duplicates — see pricing — which is the last realistic chance to catch a collision before the bank’s memory does it for you.