Ollama: Self-Hosted Large Language Models Replacing OpenAI and Claude APIs Training Course
Ollama is an open-source tool for running large language models locally on consumer and enterprise hardware. It abstracts model quantization, GPU allocation, and API serving into a single command-line interface, enabling organizations to self-host LLMs like Llama, Mistral, and Qwen without sending prompts or data to OpenAI, Anthropic, or Google.
This instructor-led, live training (online or onsite) is aimed at intermediate AI engineers and platform operators who wish to use Ollama to replace cloud LLM APIs with self-hosted, sovereign language model inference.
By the end of this training, participants will be able to:
- Install Ollama on Linux, macOS, and Windows with GPU support.
- Pull, quantize, and serve models from the Ollama registry and HuggingFace.
- Build custom Modelfiles with system prompts and parameter tuning.
- Integrate local LLMs with applications via the OpenAI-compatible API.
- Optimize inference performance for CPU-only and multi-GPU setups.
Format of the Course
- Interactive lecture and discussion.
- Lots of exercises and practice.
- Hands-on implementation in a live-lab environment.
Course Customization Options
- To request a customized training for this course, please contact us to arrange.
Course Outline
AI Sovereignty and LLM Local Deployment
- Risks of cloud LLMs: data retention, training on inputs, foreign jurisdiction.
- Ollama architecture: model server, registry, and OpenAI-compatible API.
- Comparison with vLLM, llama.cpp, and Text Generation Inference.
- Model licensing: Llama, Mistral, Qwen, and Gemma terms.
Installation and Hardware Setup
- Installing Ollama on Linux with CUDA and ROCm support.
- CPU-only fallback and AVX/AVX2 optimization.
- Docker deployment and persistent volume mapping.
- Multi-GPU setup and VRAM allocation strategies.
Model Management
- Pulling models from the Ollama registry: ollama pull llama3.
- Importing GGUF models from HuggingFace and TheBloke.
- Quantization levels: Q4_K_M, Q5_K_M, Q8_0 tradeoffs.
- Model switching and concurrent model loading limits.
Custom Modelfiles
- Writing Modelfile syntax: FROM, PARAMETER, SYSTEM, TEMPLATE.
- Temperature, top_p, and repeat_penalty tuning.
- System prompt engineering for role-specific behavior.
- Creating and publishing custom models to local registry.
API Integration
- OpenAI-compatible /v1/chat/completions endpoint.
- Streaming responses and JSON mode.
- Integrating with LangChain, LlamaIndex, and custom apps.
- Authentication and rate limiting with reverse proxy.
Performance Optimization
- Context window sizing and KV cache management.
- Batch inference and parallel request handling.
- CPU thread allocation and NUMA awareness.
- Monitoring GPU utilization and memory pressure.
Security and Compliance
- Network isolation for model serving endpoints.
- Input filtering and output moderation pipelines.
- Audit logging of prompts and completions.
- Model provenance and hash verification.
Requirements
- Intermediate Linux and container administration.
- Understanding of machine learning and transformer models at high level.
- Familiarity with REST APIs and JSON.
Audience
- AI engineers and developers replacing cloud LLM APIs.
- Organizations with data sensitivity preventing cloud model usage.
- Government and defense teams requiring air-gapped language models.
Open Training Courses require 5+ participants.
Ollama: Self-Hosted Large Language Models Replacing OpenAI and Claude APIs Training Course - Booking
Ollama: Self-Hosted Large Language Models Replacing OpenAI and Claude APIs Training Course - Enquiry
Ollama: Self-Hosted Large Language Models Replacing OpenAI and Claude APIs - Consultancy Enquiry
Upcoming Courses
Related Courses
Advanced Ollama Model Debugging & Evaluation
35 HoursAdvanced Ollama Model Debugging & Evaluation is an intensive course dedicated to diagnosing, testing, and measuring model behaviour within local or private Ollama deployments.
This instructor-led live training (delivered online or onsite) caters to advanced-level AI engineers, ML Ops professionals, and QA practitioners aiming to guarantee the reliability, fidelity, and operational readiness of Ollama-based models in production environments.
Upon completing this training, participants will be able to:
- Systematically debug Ollama-hosted models and reliably reproduce failure modes.
- Design and execute robust evaluation pipelines utilising both quantitative and qualitative metrics.
- Implement observability features (logs, traces, metrics) to monitor model health and detect drift.
- Automate testing, validation, and regression checks integrated into CI/CD pipelines.
Course Format
- Interactive lectures and discussions.
- Hands-on labs and debugging exercises using Ollama deployments.
- Case studies, group troubleshooting sessions, and automation workshops.
Customisation Options
- To request a bespoke training for this course, please contact us to arrange.
Building Private AI Workflows with Ollama
14 HoursThis instructor-led, live training in Malaysia (online or onsite) is aimed at advanced-level professionals who wish to implement secure and efficient AI-driven workflows using Ollama.
By the end of this training, participants will be able to:
- Deploy and configure Ollama for private AI processing.
- Integrate AI models into secure enterprise workflows.
- Optimize AI performance while maintaining data privacy.
- Automate business processes with on-premise AI capabilities.
- Ensure compliance with enterprise security and governance policies.
Deploying and Optimizing LLMs with Ollama
14 HoursThis instructor-led live training in Malaysia (online or onsite) is aimed at intermediate-level professionals who wish to deploy, optimise, and integrate LLMs using Ollama.
By the end of this training, participants will be able to:
- Set up and deploy LLMs using Ollama.
- Optimise AI models for performance and efficiency.
- Leverage GPU acceleration for improved inference speeds.
- Integrate Ollama into workflows and applications.
- Monitor and maintain AI model performance over time.
EXO: End-to-End Local AI Cluster Deployment
21 HoursThis instructor-led, live training in Malaysia (online or onsite) is aimed at system administrators and DevOps engineers who wish to deploy, configure, and manage EXO clusters for private LLM inference across multiple Apple Silicon or Linux nodes.
EXO for DevOps: Building Private AI Infrastructure
21 HoursThis instructor-led, live training in Malaysia (online or onsite) is aimed at DevOps engineers and infrastructure architects who wish to automate the provisioning, monitoring, and lifecycle management of private AI clusters built on EXO.
EXO Security and Governance: Offline Model Management
14 HoursThis instructor-led, live training in Malaysia (online or onsite) is aimed at security engineers and compliance officers who wish to harden EXO deployments, control model access, and govern AI workloads running entirely on-premise.
Fine-Tuning and Customizing AI Models on Ollama
14 HoursThis instructor-led, live training in Malaysia (online or onsite) is aimed at advanced-level professionals who wish to fine-tune and customise AI models on Ollama for enhanced performance and domain-specific applications.
By the end of this training, participants will be able to:
- Set up an efficient environment for fine-tuning AI models on Ollama.
- Prepare datasets for supervised fine-tuning and reinforcement learning.
- Optimise AI models for performance, accuracy, and efficiency.
- Deploy customised models in production environments.
- Evaluate model improvements and ensure robustness.
Secure Local Agentic AI: On-Prem Ollama Development for Regulated Industries
21 HoursThis instructor-led, live training in Malaysia (online or onsite) is designed for developers and technical teams who wish to use Ollama and open models to build and run private agentic AI solutions on internal infrastructure.
By the end of this training, participants will be able to install and configure Ollama, evaluate and run open models locally, create simple agentic and retrieval-based workflows, and apply security and governance controls for regulated environments.
Multimodal Applications with Ollama
21 HoursOllama serves as a platform for running and fine-tuning large language and multimodal models directly on your local machine.
Delivered via instructor-led live training (available online or onsite), this programme is tailored for advanced-level machine learning engineers, AI researchers, and product developers seeking to construct and deploy multimodal applications leveraging Ollama.
Upon completion of this training, participants will be equipped to:
- Deploy and operate multimodal models using Ollama.
- Combine text, image, and audio inputs for practical, real-world solutions.
- Develop systems for document comprehension and visual question answering.
- Create multimodal agents capable of reasoning across different data modalities.
Training Format
- Interactive lectures and discussions.
- Practical exercises using real multimodal datasets.
- Live laboratory sessions for implementing multimodal pipelines with Ollama.
Customization Options
- For a tailored training experience, please reach out to us to arrange your custom session.
Getting Started with Ollama: Running Local AI Models
7 HoursThis instructor-led, live training session in Malaysia (online or onsite) is designed for beginner-level professionals eager to install, configure, and utilise Ollama to run AI models on their own devices.
Upon completion of this training, participants will be able to:
- Grasp the fundamentals of Ollama and its key capabilities.
- Configure Ollama to run local AI models.
- Deploy and interact with LLMs using Ollama.
- Optimise performance and resource allocation for AI workloads.
- Explore practical applications for local AI deployment across various industries.
Ollama & Data Privacy: Secure Deployment Patterns
14 HoursOllama is a platform that enables the local execution of large language and multimodal models while supporting secure deployment strategies.
This instructor-led live training, available online or on-site, targets intermediate-level professionals aiming to deploy Ollama with robust data privacy and regulatory compliance measures.
Upon completing this training, participants will be able to:
- Deploy Ollama securely within containerized and on-premises environments.
- Apply differential privacy techniques to protect sensitive data.
- Implement secure logging, monitoring, and auditing practices.
- Enforce data access controls that align with compliance requirements.
Course Format
- Interactive lectures and discussions.
- Hands-on labs focused on secure deployment patterns.
- Compliance-focused case studies and practical exercises.
Customization Options
- To request customized training for this course, please contact us to arrange.
Ollama Applications in Finance
14 HoursOllama is a lightweight platform designed for running large language models locally.
This instructor-led, live training (available online or onsite) targets intermediate-level finance professionals and IT staff looking to implement, customize, and operationalize Ollama-based AI solutions within financial environments.
Upon completing this training, participants will acquire the skills necessary to:
- Deploy and configure Ollama for secure application in financial operations.
- Integrate local LLMs into analytical and reporting workflows.
- Adapt models to finance-specific terminology and tasks.
- Apply security, privacy, and compliance best practices.
Course Format
- Interactive lecture and discussion.
- Hands-on exercises using financial data.
- Live-lab implementation of finance-focused scenarios.
Course Customization Options
- To request a customized training for this course, please contact us to arrange.
Ollama Applications in Healthcare
14 HoursOllama is a lightweight platform designed for running large language models locally.
This instructor-led, live training session (available online or on-site) is designed for intermediate-level healthcare practitioners and IT teams seeking to deploy, customize, and operationalize Ollama-based AI solutions within clinical and administrative settings.
Upon completing this training, participants will be able to:
- Install and configure Ollama to ensure secure usage in healthcare environments.
- Integrate local Large Language Models (LLMs) into clinical workflows and administrative processes.
- Customize models to handle healthcare-specific terminology and tasks.
- Apply best practices for privacy, security, and regulatory compliance.
Course Format
- Interactive lectures and discussions.
- Hands-on demonstrations and guided exercises.
- Practical implementation within a sandboxed healthcare simulation environment.
Customization Options
- To request customized training for this course, please contact us to arrange.
Ollama for Responsible AI and Governance
14 HoursOllama serves as a platform for executing large language and multimodal models on local infrastructure, with a strong emphasis on supporting governance frameworks and responsible AI practices.
Offered as an instructor-led live training session—available either online or onsite—this course targets intermediate to advanced professionals who aim to embed fairness, transparency, and accountability into applications powered by Ollama.
Upon completion of this training, participants will be equipped to:
- Integrate responsible AI principles into Ollama deployments.
- Deploy strategies for content filtering and bias mitigation.
- Architect governance workflows that ensure AI alignment and auditability.
- Set up monitoring and reporting frameworks to meet compliance requirements.
Course Format
- Engaging interactive lectures and discussions.
- Practical labs focused on designing governance workflows.
- Real-world case studies and exercises centered on compliance.
Customization Options
- For organizations seeking tailored training for this course, please reach out to us to arrange specific requirements.
Sovereign AI for Regulated Organizations: Controlling Data, Models and Inference Environments
7 HoursThis instructor-led live training in Malaysia (online or onsite) is designed for intermediate-level IT leaders, compliance professionals, security teams, and enterprise architects who wish to apply sovereign AI principles and governance practices. Participants will learn to design AI environments that protect sensitive data, fulfil localisation obligations, and mitigate vendor lock-in.
By the end of this training, participants will be able to explain sovereign AI concepts, evaluate hosting and governance options, define controls for prompts and logs, and create a practical adoption roadmap.
