Diploma in Evaluation, Security and Observability in NLP
Online postgraduate diploma of 800 hours in evaluation, security and observability in NLP, structured in 8 modules covering language model evaluation, quality and bias, NLP application security, red teaming, production observability, data privacy and AI governance, with an assigned technical tutor.
Diploma in Evaluation, Security and Observability in NLP — studying from South Africa
South Africa has a growing digital economy, with strong demand for data scientists, analysts and simulation specialists in finance, mining, telecoms and government. Major banks, mining houses and telecommunications operators such as MTN and Vodacom rely on advanced analytics for planning and operations. Universities including the University of Cape Town and Wits contribute research in machine learning and computational modelling. Protection of personal data is governed by national legislation, which shapes how analytics work is carried out.
Fees for South Africa: R 27 900 (South African rand), with instalments and employer sponsorship available.
Relevant bodies in South Africa
Typical roles
Questions from South Africa
Is a SEIUM qualification recognised by any South African professional body for data and AI?
South Africa has no single statutory register for data scientists or AI practitioners, and bodies such as the Computer Society of South Africa and the South African Qualifications Authority set their own standards. SEIUM is a private online institute whose qualifications are not state degrees and are not accredited by a national body. Recognition is decided by employers or the relevant body, so ask them directly before enrolling.
Do South African employers in data and AI value private online qualifications?
Demand for data and analytics skills in South Africa spans banking, insurance, telecoms, retail and government, and employers often assess skills through portfolios, technical interviews and experience. A private online qualification may support your CV, but it does not guarantee a job or salary outcome. Show prospective employers the syllabus and your project work to gauge their interest.
How does the workload fit with a job, and what English level do I need?
The programme is fully online and taught in English, so you need confident reading and writing skills. As it is remote, no student visa is needed to study from home. Master's programmes total 1,900 hours, postgraduate diplomas 800 hours and specialist courses 300 hours, with an assigned technical tutor and a final project. Fees can be paid in instalments, and employer sponsorship may be possible.
Take the controls.
Run a real engineering trade-off from this field. Move the sliders, watch the model react in real time and beat the challenge.
What you will be able to do
Evaluate language models with automatic metrics, human evaluation and LLM-as-a-judge.
Measure bias, toxicity and robustness in NLP applications.
Identify vulnerabilities according to the OWASP Top 10 for LLM and MITRE ATLAS.
Plan and run red teaming exercises.
Implement observability and continuous evaluation in production.
Protect personal data across the life cycle of NLP applications.
Prepare compliance documentation in line with the AI Act.
Who it's for
- Model evaluation engineers looking to deepen their expertise in NLP evaluation, security and observability.
- AI security specialists moving into language model applications.
- MLOps engineers working on LLMs in production.
- AI systems auditors.
- AI compliance officers.
- NLP researchers.
- Engineering or science graduates seeking to enter banking and insurance.
What you'll work with
- Regulation (EU) 2024/1689 (Artificial Intelligence Act)
- ISO/IEC 42001:2023 (AI management systems)
- NIST AI 100-1 (AI Risk Management Framework)
- ISO/IEC 23894:2023 (AI risk management)
- Regulation (EU) 2016/679 (General Data Protection Regulation)
- ISO/IEC TR 24027:2021 (bias in AI systems)
- NIST AI 600-1 (generative AI profile)
- ISO/IEC 27001:2022 (information security)
- lm-evaluation-harness
- Garak
- Langfuse
- Microsoft Presidio
- NeMo Guardrails
8 modules
1.1NLP tasks and classic metrics1.2BLEU, ROUGE, METEOR and BERTScore1.3Perplexity and its limitations1.4Benchmarks: MMLU, HELM and BIG-bench1.5Evaluation data contamination1.6LLM-as-a-judge evaluation1.7Human evaluation and inter-annotator agreement1.8Designing domain test sets1.9Statistical significance in comparisons1.10Workshop: comparing two models with lm-evaluation-harness
2.1Hallucinations and factuality2.2Faithfulness metrics with respect to sources2.3Social biases in language models2.4Bias benchmarks: BBQ and StereoSet2.5Toxicity and offensive language2.6Robustness to input variations2.7Calibration and response confidence2.8Multilingual and Spanish-language evaluation2.9Documentation with model cards2.10Practical exercise: bias audit of a classifier
3.1OWASP Top 10 for LLM applications3.2Direct and indirect prompt injection3.3Jailbreaks and safeguard evasion3.4Training data leakage3.5Data poisoning and backdoors3.6Security of agents and tools3.7Guardrails with NeMo Guardrails and Llama Guard3.8Input and output filtering3.9MITRE ATLAS for AI threats3.10Lab: prompt injection testing
4.1Planning red teaming exercises4.2Automatic attack generation4.3Tools: Garak and PyRIT4.4Adversarial attacks on text classifiers4.5Privacy and extraction testing4.6Domain-specific abuse scenarios4.7Logging and classifying findings4.8Mitigation and re-evaluation4.9Communicating residual risks4.10Case study: red teaming a support chatbot
5.1Tracing prompts, context and responses5.2OpenTelemetry for LLM applications5.3Platforms: Langfuse, Arize Phoenix and LangSmith5.4Latency, cost and token metrics5.5Semantic drift detection5.6Continuous online evaluation5.7User feedback and labelling5.8Quality and security alerts5.9AI incident management5.10Workshop: traces and dashboards with Langfuse
6.1Personal data in prompts and logs6.2PII detection and anonymisation with Presidio6.3Differential privacy in training6.4Data minimisation and retention6.5Data protection impact assessments6.6International transfers and providers6.7Data subject rights6.8Federated learning and distributed data6.9Contracts with model providers6.10Practical exercise: anonymising conversations with Presidio
7.1European AI Act: risk levels7.2Obligations for general-purpose models7.3Management systems with ISO/IEC 420017.4Risk management with ISO/IEC 238947.5The NIST AI RMF framework7.6Technical documentation and traceability7.7Transparency and user notices7.8Internal AI audits7.9Ethics committees and human oversight7.10Case study: compliance dossier for an assistant
8.1Selecting the NLP application8.2Evaluation plan and metrics8.3Quality and bias evaluation8.4Red teaming and security testing8.5Guardrail design8.6Observability in production8.7Compliance dossier8.8Residual risk report8.9Report and presentation8.10Final project: final review with the tutor
How you'll learn
The programme is delivered 100% online on the SEIUM campus, with downloadable material, recorded classes and an assigned technical tutor. There are no fixed timetables: you progress at your own pace within the intake, making it compatible with work.
Each module combines theory, guided reading of the original standard and applied exercises. The working software is lm-evaluation-harness, Garak, Langfuse, Microsoft Presidio, NeMo Guardrails, in free, academic or evaluation versions. Tests are studied on real procedures: comparative benchmarking of models on domain sets, red teaming exercises on conversational assistants, and quality and security monitoring in production. Assessment is continuous by module plus the final capstone project.
Standards covered: Regulation (EU) 2024/1689 on Artificial Intelligence, ISO/IEC 42001:2023, NIST AI 100-1, ISO/IEC 23894:2023, Regulation (EU) 2016/679 (GDPR), ISO/IEC TR 24027:2021, NIST AI 600-1, ISO/IEC 27001:2022.
Build it end to end
Your final project follows the real engineering workflow, step by step, reviewed by your tutor.
Where this takes you
Requirements
- Prior technical training or professional experience in the sector. No university degree is required.
- An internet connection and a computer capable of running the programme's software.
- English sufficient to read technical standards.
Ready to start?
Send your application today. Our admissions team replies within 72 hours with your study plan and funding options.
Frequently asked questions
The programme totals 800 hours over 6–8 months. You set your own pace within each intake, and your tutor helps you plan the workload around your job.
You work with lm-evaluation-harness, Garak, Langfuse, Microsoft Presidio and NeMo Guardrails, using free, academic or evaluation versions. Your tutor indicates the free alternative wherever one exists.
A SEIUM Institute diploma, awarded on passing the continuous assessment and the final project. It is SEIUM's own qualification, not an official state degree; its recognition depends on the programme completed and the regulations of each country.
Yes. Modules passed are recognised in a higher programme in the same area (postgraduate diploma or master's). Ask about the recognition policy when you enrol.
Prior technical training or professional experience in the sector (no university degree required), a computer able to run the software, and English sufficient to read technical standards.
Start dates are set by SEIUM's faculty for each intake. Our admissions team will confirm the next available date when you apply.
Related programmes
Diploma in PdM Deployment: Integration with SCADA/CMMS
Online postgraduate diploma of 800 hours in PdM deployment and SCADA/CMMS integration. 8 modules covering SCADA and historians, CMMS/EAM integration, OT/IT architecture, predictive alert management, industrial…
Diploma in Geo-Temporal Predictive Models
Online postgraduate diploma of 800 hours in geo-temporal predictive models. 8 modules covering spatial and temporal dependence, classical statistical models, geospatial machine learning, spatio-temporal deep learning,…
Diploma in Root Cause Analytics and Data-Based Diagnostics
Online postgraduate diploma of 800 hours in root cause analytics and data-driven diagnosis. 8 modules covering classical root cause analysis, diagnostic data preparation, multivariate statistics, explainable…
More in Data, AI, Simulation & Analytics
All Data, AI, Simulation & Analytics programmes → · How to apply · Fees & financing · Scholarships