Machine Design #42: FAT/SAT for Automated Machines — “It Runs Fine” Is Not Acceptance

Machine Design #42: FAT and SAT for Automated Machines — Do Not Accept a Machine on "It Runs Fine"

The machine runs for thirty minutes. No major alarms. A few unfinished points go into the minutes with the phrase "to be handled later". Both sides sign the FAT.

At site, the air supply is different, real workpieces carry real variation, the interface to the upstream machine is not settled, and the backup does not match the version actually running. At that moment, "we already did the FAT" does not tell anyone what was actually proven.

FAT and SAT are not two ceremonial trial runs. They are gates that produce evidence that the machine meets agreed requirements under defined conditions.

An acceptance chain can be summarised like this:

Requirement → acceptance criterion → test method → result and evidence → deviation → disposition and retest → handover baseline

This article sets out how to build a traceable, verifiable FAT/SAT protocol for an automated machine. It is not a checklist sufficient for every machine, every contract or every market. Electrical verification, machinery risk assessment and safety validation still follow their own standards, specifications and authorities.

1. Separate FAT, FIT, SAT, SIT and commissioning

These names are often used interchangeably, and that is where scope gets lost.

FAT — Factory Acceptance Test

Testing at the manufacturer before the machine ships. The aim is to prove as much as can be proven under factory conditions.

A FAT usually covers:

Configuration and documentation.
Assembly and workmanship.
I/O, sequence and modes.
Alarms, recovery and changeover.
Performance that can be simulated or measured in the factory.
Backup, restore and the delivery baseline.

FIT — Factory Integration Test

Focused on integration between subsystems, where they can be connected in the factory:

PLC and robot.
Vision and motion.
Machine controller and line controller.
HMI, historian or middleware.
Upstream and downstream simulators.

An FIT earns its place when a single machine runs correctly but the handshake and the ownership between systems have not been proven.

SAT — Site Acceptance Test

Testing after transport, installation and connection at the place of use. The SAT confirms the as-installed machine under site conditions.

A SAT usually has to check:

The real utilities.
Foundation, levelling and environment.
Electrical, pneumatic, network and site-system connections.
Real workpieces, real operators and the real workflow.
Interfaces to the surrounding equipment.
Performance and quality under the agreed conditions.
Documentation, training and handover.

SIT — Site Integration Test

Focused on integration at site, where only there are the real systems present:

The handshake with the line.
Data exchange with MES or the production system.
Safety zoning across machines or cells, where that is in scope.
Material flow and recovery across systems.

Commissioning

Commissioning is the process of bringing the system into a correct operating state: checking the installation, configuring, tuning, testing functions and resolving problems. Some commissioning activities create the conditions or the evidence for the SAT, but the two are not the same thing.

2. Start from requirement traceability

If the protocol is written from the memory of the commissioning engineer, the requirements that are difficult or not very visible are the first ones to be missed.

Before testing, build a matrix:

Requirement ID	Requirement	Source	Gate	Test case	Evidence	Status
R-xxx	Content that can be verified	Specification, drawing, risk assessment	FAT / FIT / SAT / SIT / validation	TC-xxx	Log, measurement, record	Open / Pass / Fail

The matrix answers:

Which requirement has no test yet?
Which requirement is a given test proving?
Which requirement can only be checked at site?
Which requirement belongs to a separate safety validation?
Does a specification change invalidate an existing test case?

Not every requirement needs its own test case. One test can cover several requirements if the relationship is written down. But no important requirement should be treated as met simply because "the machine ran".

3. An acceptance criterion has to be measurable

Weak criteria:

The machine runs fine.
Appearance is OK.
Alarms work well.
Cycle time is met.
Model changeover has no errors.

None of these states the condition, the measurement method or the pass/fail boundary.

A better criterion says:

Which object.
The test condition.
The method or the data.
The expected value or behaviour.
The tolerance or the evaluation rule.
The number of runs or the sample size per the agreed specification.

An example of the structure:

With the specified model and load, run the sequence from the test case preconditions; the result must meet the time and quality criteria referenced in the specification, and must not create an unresolved deviation.

This article gives no example numbers, because takt, accuracy, sample size and capability have to come from the contract, the technical requirements or the project validation plan.

4. What fields does a test case need?

At minimum:

Test case ID and revision.
The related requirement or source.
The objective.
Gate and location.
Preconditions.
The machine configuration baseline.
Tools, test data and sample identity.
The person performing it, and a witness if required.
Test steps.
Acceptance criterion.
Actual result.
Evidence reference.
Pass or fail.
Deviation ID.
Retest requirement.

If the test uses measuring equipment, its status and traceability have to be managed to suit the project's requirements. If it uses logs or screenshots, you have to know which version, which time and which context they came from.

A photograph of a machine running does not prove cycle time. A video with no timestamp, or one not linked to a test case, also makes poor evidence.

5. Preconditions matter as much as the result

Two runs of the same sequence under different preconditions can produce results that cannot be compared.

Preconditions may include:

Software and configuration revision.
Mode and the initial machine state.
Model or recipe.
Workpiece type and condition.
Utilities.
Temperature or environment.
Tooling and change parts.
Sensor and calibration state.
Upstream and downstream simulators.
User role and access rights.

If the preconditions are not recorded, a passing test may only be true for a state that was adjusted by hand, and may not repeat after a reboot or a changeover.

6. What to check at the FAT

The FAT is the opportunity to fix problems while the machine is still at the manufacturer. Do not spend all of it on a well-rehearsed demonstration sequence.

Configuration baseline

Hardware and BOM revision.
Drawing and schematic revision.
PLC, HMI, robot and vision software revisions.
Parameter and recipe baseline.
Backup package and restore instructions.
The list of licences or dependencies in scope.

Passwords, PLC addresses and network topology do not belong in widely distributed minutes. Sensitive access data has to be handed over through a controlled channel of its own.

Assembly and inspection

Mechanics, covers, guards and access.
Cable and hose routing.
Identification and labelling per the specification.
Fastener and adjustment marks where needed.
Maintenance access and replaceability.
Cleanliness and workmanship.

Functional sequence

Auto, manual, setup and maintenance modes, as far as they are in scope.
Start, stop, pause, resume.
The normal sequence.
Empty running and running with material present.
Model changeover.
Restart and recovery after an interruption.

Diagnostics

Alarm messages.
The first-out condition.
Context snapshots.
Acknowledge, condition clear and reset.
History and logging.
Role-based access, where it exists.

Representative failures

Inside the test plan and under control:

Missing or invalid feedback.
Representative timeouts.
Model or change-part mismatch.
Loss of a utility, per an approved scenario.
Communication interruption at a suitable interface.

Fault injection is not something to improvise, and it is never done by defeating a safeguard to save time.

Preliminary performance

Cycle behaviour.
Repeatability.
Output quality.
Thermal and run-duration behaviour.
Changeover time, where it is a requirement.

FAT performance only means something under the factory conditions that were recorded. It does not automatically replace a SAT with the real utilities, the real workpieces and the real systems.

7. What a FAT cannot prove

A good FAT also states its own limits.

For example:

The real voltage, air and utilities at site.
Site foundation and vibration.
Network policy or site infrastructure.
The real MES or line controller.
Upstream and downstream equipment that is not present yet.
Production workpieces with their full variation.
Operator organisation and material logistics.
The real heat, dust, light or electrical noise.
Long-run performance against the production schedule.

Every unproven item has to be routed to the SAT, the SIT, a production trial or another validation. None of them should sit in a "check it later" state with no owner and no gate.

8. Before shipping: close the delivery baseline

There are usually modifications after the FAT. Without closing a baseline, the machine that ships can differ from the machine that was tested.

The baseline should include:

Software and configuration versions.
A backup taken after the last change.
A hash or another suitable integrity check.
Drawing, BOM and manual revisions.
The parameter and recipe release.
Open deviations.
Approved temporary dispositions.
Retest results.
Packing and shipping state.

If something is changed after the FAT but before shipping:

Record the change.
Assess the impact.
Choose the regression and retest scope.
Update the evidence.
Regenerate the baseline.

"It was only one small line" is not an automatic reason to skip the retest.

9. The SAT starts from the as-installed condition

Before running production tests, confirm the machine after transport and installation:

Damage or movement marks.
Levelling, anchoring and alignment.
Guards, access and clearance.
Utilities and connections.
Direction and phase, where applicable.
Cables, hoses and site interfaces.
Environmental conditions.
The backup and configuration actually running.
Open items from the FAT.

A SAT should not begin by pressing Auto before anyone knows whether the machine baseline and the site conditions are correct.

10. Site performance needs agreed conditions

Cycle time or quality only mean something when both sides have agreed:

The product or model mix.
Material condition.
Sample size or duration.
Which time is excluded and which is included.
Operator actions.
Upstream and downstream availability.
The rework and retry rule.
The quality measurement method.
Stop classification.

If an hour of running includes a stop caused by missing material, cycle performance cannot be argued from total output alone. The protocol has to define in advance how the data is classified and handled.

Criteria should not be re-chosen after the results are visible.

11. FIT and SIT are where ownership faults surface

Integration failures rarely sit entirely inside one subsystem:

Who sends the request?
Who holds the state?
Whose timeout is it?
Does a retry create a duplicate action?
On loss of communication, what state does each side go to?
On reconnection, how is state resynchronised?
Where does the originating alarm appear?

FIT and SIT should test both the normal handshake and a representative set of abnormal sequences. Evidence can include:

State and sequence traces.
Timestamped interface logs.
Alarm history.
Recovery results.
The version on both ends of the interface.

Sensitive tags, addresses or topology do not belong in a publicly distributed document.

12. Safety validation is not one line in the FAT

The phrase "Safety: OK" does not prove that the specified safety functions have been validated.

Safety validation follows its own route:

The risk assessment.
Safety requirements and specification.
The corresponding architecture and calculation.
Analysis and test procedures.
The applicable standard, for example ISO 13849-2 where relevant.
A person with the authority to perform and review it.
Traceable records and results.

Some of those tests may be witnessed during the FAT or SAT, but the safety validation file keeps its own scope, method and evidence. A FAT sign-off does not replace verification or validation of risk reduction.

Electrical verification follows the applicable design and standards in the same way, for example IEC 60204-1. It is not compressed into a line reading "electrical OK".

13. Manage deviations: do not let "pass with note" live forever

Every deviation needs:

An ID.
The related test or requirement.
A description of actual against expected.
The impact on safety, quality, performance, maintainability and schedule.
An owner.
A due date.
A disposition.
A temporary control, where one is permitted.
The retest scope.
Closure evidence.

Dispositions can differ:

Fix before testing continues.
Accept under an approved change.
Move to the SAT under stated conditions.
Reject.

Not every deviation blocks shipment, but the decision has to be transparent and made by someone with the authority to make it. "Pass with note" is not a disposition if nobody owns the next step.

14. Retest according to impact

After a fix:

Retest the fault directly.
Look at functions that share the component or the logic.
Look at the upstream and downstream interfaces.
Look at alarms and recovery.
Look at other modes.
Look at requirements that already passed but depend on what was just changed.

For example, changing a timeout to remove a nuisance alarm can affect cycle time, fault detection and recovery. Retesting only that one alarm line is not enough.

An impact matrix helps choose the regression scope, instead of retesting the whole machine or testing too narrowly.

15. Golden samples and test data also need a revision

A golden sample with no identity can be swapped, worn or modified without anyone noticing.

What has to be managed:

Sample ID.
Revision or model.
The characteristic it is used to test.
Condition and storage.
The date and the owner who confirmed it.
When it must be replaced or re-qualified.

Test data, image sets, recipes and simulators are the same. If the dataset changes after the FAT, the earlier results are no longer automatically equivalent.

16. Handover is more than handing over a USB stick

A handover can include:

The released software and configuration backup.
Restore and recovery instructions.
Drawings, BOM and manuals.
Parameter and recipe governance.
The alarm list and the troubleshooting route.
Risk assessment and validation records, subject to access rights.
FAT, SAT, deviation and retest records.
Spare and wear-part lists.
The maintenance plan.
Training records.
Known limitations and open items.
The support and escalation channel.

Credentials, keys and access information go through a secure channel — never inside a public document or a widely shared folder.

Most important of all: the owner has to have a way to prove that the files handed over are the baseline actually running on the machine.

17. Checklist for a FAT/SAT protocol

Does every important requirement have a gate and a test or evidence route?
Are FAT, FIT, SAT, SIT and safety validation separated in scope?
Is every acceptance criterion measurable?
Does each test case carry preconditions and a baseline?
Do tools, samples and test data have a suitable identity and revision?
Is the actual result recorded, not just a Pass tick?
Does the evidence trace back to the test case?
Has everything the FAT could not prove been routed to a later gate?
Do post-FAT changes have an impact assessment and a retest?
Does each deviation have an owner, a due date, a disposition and closure evidence?
Does the SAT check the as-installed condition before performance?
Does site performance have agreed model, material, duration and data rules?
Do safety and electrical verification have their own records per the applicable standards?
Do the backup, configuration and drawing baselines match the machine as delivered?
Does the handover include the restore method, the open items and appropriate access rights?

Conclusion

A good FAT and SAT do not try to prove everything in one session. They make six things clear:

Which requirement is being tested?
Which gate has the conditions to test it?
On what criterion is pass or fail decided?
What evidence is retained?
Who handles a deviation, and how is it retested?
Which baseline is actually delivered?

Once requirement, test case, evidence, deviation and baseline are joined into one traceable chain, "the machine runs fine" stops being an acceptance criterion. It is replaced by a body of evidence that both the manufacturer and the owner can use to operate, maintain and manage change over the long term.

Public references

IEC 62381:2024 — FAT, FIT, SAT and SIT for automation systems: https://webstore.iec.ch/en/publication/67572
IEC 60204-1:2016 + AMD1:2021 — Electrical equipment of machines: https://webstore.iec.ch/en/publication/26037
ISO 12100:2010 — Machinery risk assessment and risk reduction: https://www.iso.org/standard/51528.html
ISO 13849-2:2012 — Validation of safety-related parts of control systems: https://www.iso.org/standard/53640.html

View all MINATA technical articles