- Akamai Cloud account with Akamai Functions access
- Akamai Cloud personal access token
- Read and write access for Linodes and Firewalls
- Read access for Events
- Terraform installed locally
- Spin CLI installed locally
- aka plugin for Spin
- Node.js installed locally
- SSH key pair
Before you start
You’ll connect to the GPU instance over SSH while validating the Ollama deployment, so confirm that your public key is available before provisioning the infrastructure.
Build the project
Provision the inference back end first and then configure, test, and deploy the edge application.
Clone the sample application and review its application and infrastructure directories.
Clone the project repository:
git clone https://github.com/akamai-developers/akamai-functions-llm-chatbot.git
cd akamai-functions-llm-chatbot
The repository contains two primary parts:
- An infrastructure directory containing the Terraform and cloud-init configuration
- A Spin application containing the Astro front end and TypeScript API
Review the Terraform variables before provisioning. You can adjust the region, GPU plan, model, and related infrastructure settings for your environment.
Use Terraform to create the GPU instance and configure the Ollama inference service.
Move into the infrastructure directory and initialize Terraform:
cd infrastructure
terraform init
Set your Akamai Cloud personal access token:
export LINODE_TOKEN="your-pat-here"
Provision the infrastructure:
terraform apply
Terraform creates a dedicated GPU instance and uses cloud-init to:
- Install the required GPU drivers
- Install Ollama
- Pull the configured open source model
- Expose the Ollama API on port 11434
- Wait for the model to become available
When provisioning finishes, Terraform returns the Ollama endpoint:
ollama_endpoint = "http://<linode-ip>:11434"
Provisioning and model loading typically take several minutes.
Connect the edge application to the Ollama endpoint and protect its cache-management API.
Install the Spin CLI, Node.js, and the aka plugin before continuing.
Install or update the Akamai Functions plugin:
spin plugins update
spin plugins install aka
The application uses two deployment variables:
- ollama_url connects the application to the GPU-hosted Ollama API.
- secret_sauce protects the endpoint used to clear the response cache.
Generate a strong random value for secret_sauce. Requests to the cache-management endpoint must provide this value through the X-SecretSauce HTTP header.
Do not commit tokens, secrets, or production endpoint credentials to the repository.
Run the Spin application locally while using the same remote inference back end.
Start the application and pass the required variables:
spin up --build \
--variable ollama_url="http://<linode-ip>:11434" \
--variable secret_sauce="<strong-random-secret>"
Open the local application in your browser and submit a test message.
Verify that:
- The chatbot returns a model-generated answer.
- Conversation history persists through the session.
- Repeated questions can be returned from the cache.
- Requests that violate the guardrails are rejected.
- The browser does not control or submit the complete conversation history.
Using the remote Ollama back end during local development lets you test model behavior without requiring a local GPU.
Compile the application to WebAssembly and deploy it to Akamai Functions.
Confirm that Akamai Functions access is enabled for your account before deploying. If you have not been approved yet, request access and continue with the local build while your request is reviewed.
Authenticate with Akamai Functions:
spin aka login
Build and deploy the application:
spin aka deploy --build \
--variable ollama_url="http://<linode-ip>:11434" \
--variable secret_sauce="<strong-random-secret>"
After deployment, open the generated application URL and submit a new question.
The deployed application serves its front end and API from Akamai Functions. Requests are validated at the edge before the application checks the key-value store or contacts the GPU inference back end.
Confirm that edge processing reduces unnecessary requests to the model.
Test the application with both new and repeated questions.
For a new question, the application:
- Validates the incoming message.
- Applies input and topic guardrails.
- Loads the session history.
- Checks the response cache.
- Sends the request to Ollama when no cached response exists.
- Stores the answer for future requests.
- Updates the session history.
For an equivalent repeated question, the application can return the cached response directly from the edge without using another GPU inference cycle.
When testing is complete, remove infrastructure you no longer need:
cd infrastructure
terraform destroy
Dedicated GPU resources continue to generate charges until they are deleted.
Frequently Asked Questions (FAQ)
Frequently Asked Questions (FAQ)
The open source model is served by Ollama on an Akamai Cloud instance with a dedicated NVIDIA GPU. The model does not run inside Akamai Functions.
The Spin application compiles to WebAssembly for fast start-up, strong workload isolation, and deployment across Akamai’s distributed edge infrastructure.
The application normalizes each question and checks the Akamai Functions key-value store for an existing answer. A cache hit returns the saved response without sending another request to the GPU.
Conversation history is stored server-side in the Akamai Functions key-value store and associated with a session identifier in a secure cookie. The sample limits history to 20 messages.
The sample checks for prompt injection, secondary-request pivots, and code-generation requests are included. It also limits message length and uses a system prompt to keep responses within the application’s intended domain.
Yes. You can change the Ollama model through the infrastructure configuration, subject to the model’s hardware and memory requirements.
The offer applies only to eligible cloud services. Dedicated GPU services are excluded, so review the current promotional terms before provisioning the project.
Yes. The TypeScript API uses LangChain.js and an Ollama-compatible endpoint, so the static context in the sample can be extended with a document loader, retriever, and vector store.
Run terraform destroy after completing the project and confirm that the GPU instance and associated resources have been deleted from your account.